DSPT-4967 provide alternate method for encoding and passing ext JSON

[drstevens]

Attempt to define an alternate method for Java implementation for passing ext and slot ext JSON without modifying original bid request content.

See README.md update

If sending via HTTP header, currently only available in Java implementation

1. retrieve protobuf generated message from Response, Response.getExtProto
2. Use this to generate a URI safe string using ResponseUtil.encodedResponseMetadata
3. Send this string as value for HTTP Header named XAmazonTest: <encoded string>
   The goal is to avoid parsing the bid request string a second time just to insert ext JSON, only to serialize it back to bytes to send in HTTP Request body.

Unfortunately this meant making the ext JSON creation lazy and providing enough information to Response and Slot to generate both encodings.

Migrate PR from amzn/DynamicTrafficEngine#4

[drstevens]

@iabmayank Is there anything I can do to help get this merged?

I also have questions about the general future of this project. Will owners eventually publish artifacts (jar for Java), or are there no plans for this and it's up to us?

No objections either way, but we're just trying to consider the best path forward on our side knowing we need to consume this as a jar in our own projects. The answer to this question, along with potential timeline might change how we approach this.

[kchiang6997]

lgtm

[kchiang6997]

Reviewing this again, I may have missed the fact that the response structure will not be in the same ext.amazontest.learning/decision format, in which case this would need to be part of a larger release and communications sent out to SSPs to adjust their response handling.

Alternative would be to add another initializer with flag to use the new response structure instead, so changes are backward compatible.

[kchiang6997]

Also, I still don't have permissions to merge changes.

[drstevens]

Hi @kchiang6997, thanks for taking another look. Much appreciated.

I think I may be confused or didn't properly convey my proposal. If I've created this PR correctly, it's up to the SSP to send the equivalent of ext fields either the traditional way via the bid request JSON or via a header. My hope was that on your side, you would just look in both places.

I may have missed the fact that the response structure will not be in the same ext.amazontest.learning/decision format,

Would you please elaborate on what response structure change you're thinking of here? Are you talking about the change to com.amazon.demanddriventrafficevaluator.evaluation.evaluator.Response here?

Alternative would be to add another initializer with flag to use the new response structure instead, so changes are backward compatible.

Are you proposing a flag that controls whether Response.ext is populated instead of defining a lazy toExt method?

In addition to providing a new mechanism for encoding and sending the metadata you need, this PR also makes it easier to consume this response information ourselves. As others have noted, it would be ideal if we didn't have to parse the JSON in the response from the library. See #4. We have this same need and we're using this PR to accomplish this as well. We emit these signals as metrics that need to get fed back into our own systems. We haven't gotten to the reporting aspect of this integration yet, but we'd want to use these values on the response objects themselves instead of parsing JSON to pull them out for all the same reasons listed in #4.

How are other SSPs consuming this library? Are they copy/pasting this code into their own applications? Is there a plan to publish a jar and/or Go package for us to consume in the longer term?