Generating a dataset (tfrecord files)

[kalifou]

Hi,
Thanks for open-sourcing the code behind the benchmark.
I am trying to reproduce some results and it seems that the script 'run_evaluation.py' does generate some checkpoints & a results.txt files for each relevant model of the graph, but not that "nasbench.tfrecord" file. Could you please point out to the appropriate script? So far it appears to be missing...

Thanks & Regards
K. Rene Traore

[chrisying]

Sorry for not responding to this question earlier. I don't receive notifications on this repo anymore but I wanted to make sure this gets a response even if it is late.

I don't have the original script which generated the TFRecord file anymore but I can generally explain how it was generated.

The nasbench.tfrecord is essentially just an encoded file containing a list of UTF-8 encoded JSON-encoded ModelMetrics objects. This is the line that generates the JSON: https://github.com/google-research/nasbench/blob/master/nasbench/scripts/run_evaluation.py#L202 and this is the line that reads the NASBench file: https://github.com/google-research/nasbench/blob/master/nasbench/api.py#L149.

The code for writing a TFRecord file can be found here: https://www.tensorflow.org/tutorials/load_data/tfrecord#writing_a_tfrecord_file_2. The only small detail to remember is to encode the JSON string in UTF-8 bytes before writing to the TFRecord file.