Add external data processing and storage section #3033
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
splindsay-92
force-pushed
the
chat/external-data-processing
branch
4 times, most recently
from
81034ca to
141f0a2
Compare
splindsay-92
force-pushed
the
chat/external-data-processing
branch
from
a491a26 to
6b0ffcc
Compare
…more outside of specific ably features
…n and rename the page.
splindsay-92
force-pushed
the
chat/external-data-processing
branch
from
6b0ffcc to
2be4b41
Compare
splindsay-92
changed the title
AndyTWF
added
review-app
| @@ -0,0 +1,552 @@ | |||
| --- | |||
| title: "Extract messages via integrations" | |||
There was a problem hiding this comment.
We do already have https://ably.com/docs/guides/chat/export-chat - is this intended to replace that?
| ``` | ||
| </Code> | ||
|
|
||
| ## Understanding channel names and room names <a id="channel-names"/> |
There was a problem hiding this comment.
I think there's benefit here of encouraging users to group their chat rooms under a common namespace for simplicity, e.g gamechat:<room>
| ### Example webhook handler | ||
|
|
||
| <Code> | ||
| ```javascript |
There was a problem hiding this comment.
Does this add any value to the narrative? It's very abstract and is fairly obvious
| ``` | ||
| </Code> | ||
| ## Using queues <a id="queues"/> |
There was a problem hiding this comment.
Would bump below streaming
| await room.messages.send({ | ||
| text: responseText | ||
| }, { | ||
| extras: { |
There was a problem hiding this comment.
This approach doesn't work with Chat - metadata goes into data and headers go specifically into extras.headers
There was a problem hiding this comment.
(Something we should think about addressing)
| await room.messages.send({ | ||
| text: "Hello!", | ||
| metadata: { | ||
| intent: 'translate', |
There was a problem hiding this comment.
Would this decision be a client-side one, or something that the server would decide? If it's client side then you're up for spoofing.
Also for translation specifically on-device translation is something we'd consider
| - **Consistency**: Webhook calls that fail will lead to inconsistencies between your data storage and Ably, which can be difficult to resolve. Detect if this happens using the `[meta]log` channel and use the [history endpoint](#history-endpoint) to backfill missing data. | ||
| - **[At-least-once delivery](/docs/platform/architecture/idempotency#protocol-support-for-exactly-once-delivery)**: You need to handle duplicate messages. Deduplication can be done by checking `serial` and `version.serial`. | ||
|
|
||
| ## Using outbound streaming <a id="outbound-streaming"/> |
There was a problem hiding this comment.
Some of these sections feel like repeating the above?
Description
FTF-443
Originally this was just about adding a guide on data processing, but it's since expanded into a rework of the existing data-storage use-case guide into an expanded feature set under the Chat document root directory.
We now have a seperate section that deals with
external storage and processing, under which data extraction, processing and storage all sit under.Checklist