Allow per-format compression - #26
Open
gouttegd wants to merge 4 commits into
Open
Conversation
Replace the existing `project.export_formats` configuration key, which
currently accepts a simple list of formats, by a more complex key that
accepts a list of "ExportSpecification" object, where each such object
indicates both (1) the export format to use and (2) the compressions to
use.
For example:
```yaml
export_formats:
- format: owl
compressions: [none, gz]
- format: db
compressions: [gz]
```
The above means that the release artefacts should be exported in
* uncompressed OWL,
* GZip-compressed OWL,
* GZip-compressed SemSQL.
When `compressions` is not explicitly specified, the default is `[none]`
(i.e., uncompressed) for all formats, except for SemSQL where the
default is `[gz]`.
For backwards compatibility, an "old-style" (up to ODK 1.6 included)
`export_formats` declaration like this:
```yaml
export_formats:
- owl
- obo
- json
```
is automatically transformed into:
```yaml
export_formats:
- format: owl
- format: obo
- format: json
```
If the `gzip_main` option was used, this is transformed into
```yaml
export_formats:
- format: owl
compressions: [none, gz]
- format: obo
compressions: [none, gz]
- format: json
compressions: [none, gz]
```
Of note, the conversion of `gzip_main` only applies if the
`export_formats` key was not already in the "new style". If it was in
the "new style", then it is assumed that the user already specified
whatever compressions they wanted there, and `gzip_main` is simply
ignored.
We update the templates to reflect the changes in the model regarding
the list of export formats and the support for per-format compression.
This means mostly that:
(A) Whenever we need to get the formats, we need to do either
{% for spec in project.export_formats %}
The format is {{ spec.format }}.
Compressions: {% for comp in spec.compressions %}{{ comp }} {%endfor %}
{% endfor %}
or, if we don't need the compressions, we can use the convenience
auto-generated field `project.export_format_names`:
{% for name in project.export_format_names %}
The format is {{ name }}.
{% endfor %}
(B) The definition of `MAIN_FILES` in the Makefile template is changed
to include each release artefact in each format with all its available
compression formats (including no-compression).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR replaces the
gzip_mainboolean option and theexport_formatslist of export formats by a new form ofexport_formatsin which the list values can indicate, for any given format, the compressions to use.That is, it allows to do something like this:
to indicate that (i) release artefacts should be exported in OWL and SemSQL, (ii) the OWL files should be published both in uncompressed (
none) and GZip-compressed (gz) forms, (iii) the SemSQL files should be published only in GZip-compressed form.When
compressionsis not specified for a given export format, the default is (for now)nonefor most formats except for SemSQL where the default isgz. That is, this:is equivalent to
The point of this feature is to give users a finer control (compared to
gzip_main) over how the release artefacts should be compressed, as envisioned in this issue and in particular in this comment.Only no-compression (
none) and Gzip compressions (gz) are supported for now, but the new system would make reasonably trivial to add other compression formats (e.g. BZip2, XZ, etc.) in the future.Backwards compatibility with existing configuration files is preserved (and here, in front of his computer, @matentzn is breathing again ;) ). An “old-style”
export_formatsdeclaration like this:is automatically converted into
or, if
gzip_mainwas also used, intocloses INCATools/ontology-development-kit#1298