[cmd/schemagen] Extend schemagen script with ability to handle external refs.

[jkoronaAtCisco]

Context

The schemagen script was introduced in #44950. It generates schemas for OpenTelemetry Collector components configuration by analyzing Go struct definitions. The addition of this tool is part of a broader plan to permanently introduce configuration schemas to the collector. In short, it can be summarized as follows:

1. Implement a schema generation script as an temporal solution.
2. Generate schemas for collector components and shared packages
3. Add schema validation in build time (with checkapi)
4. Implement tool to generate Go configs out of schema
5. Make schemas single source of truth for components

See this discussion #42214 and RFC open-telemetry/opentelemetry-collector#13784 for more details.

Description

Following new features were added to schemagen script:

1. Allow referencing external schemas, e.g. confighttp
2. Parse all package types to single schema
3. Parse internal packages, like metadata
4. Introduced settings file
5. Support Optional generic type
6. Get types description from the comments
7. Mark go types like int8, or float64

Examples

Please find following schemas generated with schemagen:

• several common package schamas in <repo>/schemas folder
• ICMP Check Receiver config schema in <repo>/receiver/icmpcheckreceiver/config.schema.yaml

Link to tracking issue

#42214

Testing

1. Unit tests provided for script config, parser, writer and helpers.
2. Generated schemas for few common packages (in <repo>/schemas) and ICMP Check Receiver

Documentation

Documentation for the script in cmd/schemagen/README.md.

[dmitryax]

The component already exists. So it should be an enhancement right?

[dmitryax]

Can't we read the required module during parsing AST and use that instead of having the refs section?