Skip to content

Resolve "Performance"

Michelle Weidling requested to merge 9-performance into develop

This MR aims add improving the performance of the manifest.json file. The main bottleneck I identified was the fetching and rendering of the bibliographical items as well as reading the JSON file we retrieve from Zotero during the build (eupt-bibliography.json) from disk.

The countermeasures I installed are:

  • drastically shrinking the JSON file size by removing everything that is empty or not needed. This reduced the file size to about a third which makes opening and processing the file a lot faster.
  • using ujson for opening the JSON file from "disk". ujson is optimized to open and handle large JSON files.
  • prerender the bibliographical items during the build instead of rendering them on the fly

The new JSON file has the following structure:

[
  {"$KEY": "$PRERENDERED_ENTRY"},
  {"$KEY": "$PRERENDERED_ENTRY"}
]

where $KEY is the Zotero ID and $PRERENDERED_ENTRY is the HTML serialization of the bibliography entry. The metadata module now only iterates over the entries given in the main file and looks up the HTML serialization in eupt-bibliography.json.

Closes #9 (closed)

Edited by Michelle Weidling

Merge request reports