Skip to content

[India_RBIStateDomesticProduct] Made changes in the manifest file - #2180

Open
Krishnam24maheshwari wants to merge 4 commits into
datacommonsorg:masterfrom
Krishnam24maheshwari:india_rbi_sdp
Open

[India_RBIStateDomesticProduct] Made changes in the manifest file#2180
Krishnam24maheshwari wants to merge 4 commits into
datacommonsorg:masterfrom
Krishnam24maheshwari:india_rbi_sdp

Conversation

@Krishnam24maheshwari

@Krishnam24maheshwari Krishnam24maheshwari commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

In the import, Out of the 34 Statistical Variables generated in the dataset (totaling 15,435 rows), exactly 4 StatVars account for all missing reference errors.
1. RealValue_Amount_EconomicActivity_GrossValueAdded_FinancialServices
2. RealValue_Amount_EconomicActivity_NetValueAdded_FinancialServices
3. Nominal_Amount_EconomicActivity_GrossValueAdded_FinancialServices
4. Nominal_Amount_EconomicActivity_NetValueAdded_FinancialServices

In manifest.json, import_inputs does not specifies "node_mcf": "output/*.mcf".
During the pipeline execution, no .mcf definitions for these 4 variables were uploaded/passed to the genmcf stage.

Solution :
1. I have added "node_mcf" in the manifest json file so that definitions for those 5 StatVars will be available now during the genmcf stage.
2. I have raised CL to add these 4 StatVars in DC - https://critique.corp.google.com/cl/971086669 .

Validation output - https://storage.mtls.cloud.google.com/datcom-import-test/statvar_imports/india_rbistatedomesticproduct/India_RBIStateDomesticProduct/2026_09_03T08_23_34_372510_07_00/input0/validation/validation_output.csv

After remapping the statvar using -statvar_dcid_remap_csv, i am getting 2.95% deletions. "The 2.95% deletions (455 / 15,435 observations) are 100% caused by the intentional StatVar remap we just added in

statvar_remap.csv
Deleted (455 observations, 2.95%): variableMeasured: dcid:RealValue_Amount_EconomicActivity_NetDomesticProduction_PerCapita
Added (455 observations, 2.95%): variableMeasured: dcid:RealValue_Amount_EconomicActivity_NetDomesticProduction_AsAFractionOf_Count_Person
Because the differ compares the new run against the previous run (2026_02_18T01_03_27_833100_08_00), renaming the StatVar DCID causes the 455 observations under the old StatVar name (..._PerCapita) to register as deleted and the exact same 455 observations under the new remapped StatVar name (..._AsAFractionOf_Count_Person) to register as added. No actual data points were lost."

The old successful run should have no observation with the "RealValue_Amount_EconomicActivity_NetDomesticProduction_PerCapita" statvar but it was manually copied to latest_version.txt for further information - https://buganizer.corp.google.com/issues/548893664

@google-cla

google-cla Bot commented Aug 24, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the manifest.json for the India RBI State Domestic Product import by adding new parameters to the stat_var_processor.py script, tracking counter files, and adding a node_mcf field. The feedback suggests specifying the exact path for the generated MCF file in node_mcf instead of using a wildcard to prevent matching unrelated files and ensure better compatibility.

Comment thread statvar_imports/india_rbistatedomesticproduct/manifest.json
@Krishnam24maheshwari Krishnam24maheshwari changed the title Made chnages in the manifest file Made changes in the manifest file Aug 24, 2026

@Krishnam24maheshwari Krishnam24maheshwari left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done changes as per Gemini bot

Comment thread statvar_imports/india_rbistatedomesticproduct/manifest.json
@saanikaaa

Copy link
Copy Markdown
Contributor

Pls add import name in PR title. Also, aren't we adding threshold of 0.1 for this import ?

@Krishnam24maheshwari Krishnam24maheshwari changed the title Made changes in the manifest file [India_RBIStateDomesticProduct] Made changes in the manifest file Sep 1, 2026
@Krishnam24maheshwari

Copy link
Copy Markdown
Contributor Author

Pls add import name in PR title. Also, aren't we adding threshold of 0.1 for this import ?

Yes since we are no longer adding goldens that's why I didn't added the threshold part.

@Krishnam24maheshwari Krishnam24maheshwari left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added threshold

@pulkeet-wq

Copy link
Copy Markdown
Contributor

LGTM

Comment thread statvar_imports/india_rbistatedomesticproduct/manifest.json Outdated

@Krishnam24maheshwari Krishnam24maheshwari left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants