Missing metadata from LungMAP Bronchpulmonary Dysplasia (BPD)

[bvizzier-ucsc]

Reported by Bing-Xing Hua, LungMAP PI from the Broad Institute.

The LungMAP Omics Data Browser does not have a downloadable metadata file for the Bronchpulmonary Dysplasia (BPD) dataset.

Project ID from the link provided: 1977dc47-8414-4263-a870-6b0f207d8ab3

[bvizzier-ucsc]

Josh Fortriede is looking into this from the ingest side.

[hannes-ucsc]

@bvizzier-ucsc Is there an example of a LungMAP project for which you observe the expected behavior?

[bvizzier-ucsc]

1977dc47-8414-4263-a870-6b0f207d8ab3 - Added to the description.

[hannes-ucsc]

I meant a project for which you observe the expected behavior, i.e. where this is working.

[bvizzier-ucsc]

Josh confirmed that there are metadata JSON files that were submitted for ingest into TDR.

[bvizzier-ucsc]

Hmmm... I checked several datasets and couldn't find one that didn't return that message.

Bing asked Josh to identify "the JSON file" associated with this study, and Josh replied with:

A submission has hundreds or even thousands of JSON files. This submission has 2057 JSON files. The JSON files are broken down into multiple categories

• File_descriptors
  • These are for analysis files and sequence files. Each file gets a json descriptor file
• Links
  • A single file that describes all the links
• Metadata
  • Every node (HCA schema entitie: donor_organism, cell_suspension, sequence_file, process, analysis_protocol, etc). This submission has about 500 non-file entities and 800 file entities.

I did a spot check on HCA, and I cannot find a dataset there that doesn't return the same result: "The metadata is not avaialble for this project."

[bvizzier-ucsc]

Bing-Xing is no longer associated with LungMAP. I asked Joshua about this, and he said he can't help with this issue.

I suggest we close it.

[hannes-ucsc]

The Metadata section on the Metadata tab of the project details page (as shown in the description) is an artifact from HCA where it's used to expose the ingest spreadsheet. LungMAP does not use ingest spreadsheets and instead produces the metadata directly as JSON in the staging area. This section will never show anything useful on the LungMAP site. If users want all metadata, they can request a verbatim JSONL manifest using the Azul REST API. This manifest contains the original metadata in JSON form, one JSON document per line. To make this section of the UI useful, CC could update the LungMAP UI to expose that type of manifest instead.