New features must not break existing consumers

[beatgammit]

For TAP to grow, I think we need to not break existing implementations. Here's my proposal:

• Every feature request must be submitted as a pull request and have an example TAP output that exhibits corner cases of the feature as well as basic functionality
• For a feature request to be accepted, major implementations must still work (i.e. ignore the feature, have a reasonable fallback, etc)
• Some minimum number of major TAP implementations must agree to implement it once it's accepted
• Backwards compatibility only guaranteed for the previous N TAP versions (I think N should be 1 or 2)
  • this allows for some flexibility in breaking backwards compat

This should be enforced with some kind of continuous integration (travis-ci, drone.io, etc).

Does this sound reasonable? If we come up with a set of the major TAP implementations, I'd be willing to work on this.
#17 #13 #4 #3

[isaacs]

Rather than only "agree to implement once it's accepted", there should be at least one full public implementation that adds the feature prior to acceptance into the spec. (Behind a flag or otherwise opt-in is fine.)

This way, nothing gets into the spec without at least being shown to be possible and not conflict with the rest of the specification. It's too easy to agree to do something; much harder and more valuable to actually do it.

[Leont]

+1

[jonathanKingston]

@isaacs The real implementation is a great idea, certainly only in a beta or flagged mode though.

I would like all changes to go through some form of formal change process like the RFC process both Rust and Ember have:
https://github.com/emberjs/rfcs

That way the changes can be tracked, real working notes and documentation gets built and can easily be merged into the full specification / site. Ember merges the changes in once it has gone live, I suspect the same can happen with once a working code sample happens and a consensus it met.

[beatgammit]

@jonathanKingston - Would you mind if I start working on this?

In order to start, we need to decide:

• what CI we'll use
• what implementation(s) need to implement new features before it's accepted
• minimum required implementations to be in full compliance before a TAP version is released

Thoughts? I imagine there may need to be some bending of the rules for TAP 14, but after that it should be pretty consistent.