Repository navigation
Feature/3301 python cl timeout #3344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bcb37
wants to merge
6
commits into
dev
Choose a base branch
from
feature/3301-python-cl-timeout
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+719
−23
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
cb4e5c0
python cl: add timeout and null condition on mark
bcb37 6b4a907
add a python quickTest
bcb37 92638c6
use UpGradeApiError in quickTest
bcb37 394b063
Merge branch 'dev' into feature/3301-python-cl-timeout
bcb37 136a339
add awaitResult to reward methods
bcb37 20a272a
fix mark request type
bcb37 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,283 @@ | ||
| """Quick smoke test against a live UpGrade instance. | ||
|
|
||
| Run from the repo root: | ||
| python clientlibs/python/quickTest.py | ||
|
|
||
| Or from this directory: | ||
| python quickTest.py | ||
|
|
||
| Requires the library to be installed (or on PYTHONPATH). From this directory: | ||
| pip install -e . | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import asyncio | ||
| import time | ||
|
|
||
| import httpx | ||
|
|
||
| from upgrade_client_lib import UpgradeClient | ||
| from upgrade_client_lib.types.requests import LogGroupMetrics, LogInput, LogMetrics | ||
| from upgrade_client_lib.exceptions import UpgradeApiError | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Target URL — swap to point at a different environment | ||
| # --------------------------------------------------------------------------- | ||
| URL_LOCAL = "http://localhost:3030" | ||
| URL_ECS_QA = "https://apps.qa-cli.net/upgrade-service" | ||
| URL_ECS_STAGING = "https://apps.qa-cli.com/upgrade-service" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Config — edit these to match your UpGrade setup | ||
| # --------------------------------------------------------------------------- | ||
| host_url = URL_LOCAL | ||
| context = "context_identifier_1" | ||
| site = "ts" | ||
| target = "" | ||
| status = UpgradeClient.MARKED_DECISION_POINT_STATUS.CONDITION_APPLIED | ||
| feature_flag_key = "TEST_FEATURE_FLAG" | ||
|
|
||
| user_id = f"quicktest_user_{int(time.time() * 1000)}" | ||
| group: dict = {"classId": ["STORED_USER_GROUP"]} | ||
| working_group: dict = {"classId": "STORED_USER_GROUP"} | ||
| alias = f"alias_{user_id}" | ||
|
|
||
| # Reward testing — set experiment_id to a real Thompson Sampling experiment UUID | ||
| # to test send_reward by experiment id. | ||
| experiment_id = "PASTE-EXPERIMENT-UUID-HERE" | ||
| reward_value = UpgradeClient.BINARY_REWARD_VALUE.SUCCESS | ||
|
|
||
| log_inputs = [ | ||
| LogInput( | ||
| timestamp="2022-03-03T19:49:00.496", | ||
| metrics=LogMetrics( | ||
| attributes={ | ||
| "totalTimeSeconds": 41834, | ||
| "totalMasteryWorkspacesCompleted": 15, | ||
| "totalConceptBuildersCompleted": 17, | ||
| "totalMasteryWorkspacesGraduated": 15, | ||
| "totalSessions": 50, | ||
| "totalProblemsCompleted": 249, | ||
| }, | ||
| groupedMetrics=[ | ||
| LogGroupMetrics( | ||
| groupClass="conceptBuilderWorkspace", | ||
| groupKey="graphs_of_functions", | ||
| groupUniquifier="2022-02-03T19:48:53.861Z", | ||
| attributes={ | ||
| "timeSeconds": 488, | ||
| "hintCount": 2, | ||
| "errorCount": 15, | ||
| "completionCount": 1, | ||
| "workspaceCompletionStatus": "GRADUATED", | ||
| "problemsCompleted": 4, | ||
| }, | ||
| ) | ||
| ], | ||
| ), | ||
| ) | ||
| ] | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Main test flow | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| async def quick_test() -> None: | ||
| client = UpgradeClient(user_id=user_id, host_url=host_url, context=context) | ||
|
|
||
| await do_init(client) | ||
| await do_group_membership(client) | ||
| await do_working_group_membership(client) | ||
| await do_aliases(client) | ||
| await do_assign(client) | ||
| await do_assign_ignore_cache(client) | ||
| condition = await do_get_decision_point_assignment(client) | ||
| await do_feature_flags(client) | ||
| await do_feature_flags_ignore_cache(client) | ||
| await do_has_feature_flag(client) | ||
| await do_mark(client, condition) | ||
| await do_mark_no_assignment(client) | ||
| await do_send_reward_by_experiment_id(client) | ||
| await do_send_reward_by_decision_point(client) | ||
| await do_log(client) | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Individual operations | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| async def do_init(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.init() | ||
| print(f"\n[Init response]: {response}") | ||
| except Exception as error: | ||
| log_error("Init", error) | ||
|
|
||
|
|
||
| async def do_group_membership(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.set_group_membership(group) | ||
| print(f"\n[Group response]: {response}") | ||
| except Exception as error: | ||
| log_error("Group", error) | ||
|
|
||
|
|
||
| async def do_working_group_membership(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.set_working_group(working_group) | ||
| print(f"\n[Working Group response]: {response}") | ||
| except Exception as error: | ||
| log_error("Working Group", error) | ||
|
|
||
|
|
||
| async def do_aliases(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.set_alt_user_ids([alias]) | ||
| print(f"\n[Aliases response]: {response}") | ||
| except Exception as error: | ||
| log_error("Aliases", error) | ||
|
|
||
|
|
||
| async def do_assign(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.get_all_experiment_conditions() | ||
| print(f"\n[Assign response]: {response}") | ||
| except Exception as error: | ||
| log_error("Assign", error) | ||
|
|
||
|
|
||
| async def do_assign_ignore_cache(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.get_all_experiment_conditions(ignore_cache=True) | ||
| print(f"\n[Assign (ignore cache) response]: {response}") | ||
| except Exception as error: | ||
| log_error("Assign (ignore cache)", error) | ||
|
|
||
|
|
||
| async def do_get_decision_point_assignment(client: UpgradeClient) -> str | None: | ||
| try: | ||
| assignment = await client.get_decision_point_assignment(site, target) | ||
| print(f"\n[Decision Point Assignment response]: {assignment}") | ||
|
|
||
| if assignment is None: | ||
| print("\n[Decision Point Assignment]: no assignment found") | ||
| return None | ||
|
|
||
| condition = assignment.get_condition() | ||
| print(f"\n[Condition]: {condition}") | ||
|
|
||
| exp_type = assignment.get_experiment_type() | ||
| print(f"\n[Experiment Type]: {exp_type}") | ||
|
|
||
| payload = assignment.get_payload() | ||
| print(f"\n[Payload]: {payload}") | ||
|
|
||
| if payload: | ||
| print(f"\n[Payload value]: {payload.value}") | ||
|
|
||
| return condition | ||
| except Exception as error: | ||
| log_error("Decision Point Assignment", error) | ||
| return None | ||
|
|
||
|
|
||
| async def do_feature_flags(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.get_all_feature_flags() | ||
| print(f"\n[Feature Flags response]: {response}") | ||
| except Exception as error: | ||
| log_error("Feature Flags", error) | ||
|
|
||
|
|
||
| async def do_feature_flags_ignore_cache(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.get_all_feature_flags(ignore_cache=True) | ||
| print(f"\n[Feature Flags (ignore cache) response]: {response}") | ||
| except Exception as error: | ||
| log_error("Feature Flags (ignore cache)", error) | ||
|
|
||
|
|
||
| async def do_has_feature_flag(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.has_feature_flag(feature_flag_key) | ||
| print(f"\n[Has Feature Flag response]: {response}") | ||
| except Exception as error: | ||
| log_error("Has Feature Flag", error) | ||
|
|
||
|
|
||
| async def do_mark(client: UpgradeClient, condition: str | None) -> None: | ||
| try: | ||
| response = await client.mark_decision_point( | ||
| condition=condition, | ||
| status=status, | ||
| site=site, | ||
| target=target, | ||
| ) | ||
| print(f"\n[Mark response]: {response}") | ||
| except Exception as error: | ||
| log_error("Mark", error) | ||
|
|
||
|
|
||
| async def do_mark_no_assignment(client: UpgradeClient) -> None: | ||
| """Mark a decision point where no experiment is running (condition=None).""" | ||
| try: | ||
| response = await client.mark_decision_point( | ||
| condition=None, | ||
| status=UpgradeClient.MARKED_DECISION_POINT_STATUS.NO_CONDITION_ASSIGNED, | ||
| site="nonexistent-site", | ||
| target="nonexistent-target", | ||
| ) | ||
| print(f"\n[Mark (no assignment) response]: {response}") | ||
| except Exception as error: | ||
| log_error("Mark (no assignment)", error) | ||
|
|
||
|
|
||
| async def do_send_reward_by_experiment_id(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.send_reward( | ||
| reward_value=reward_value, | ||
| experiment_id=experiment_id, | ||
| ) | ||
| print(f"\n[Send Reward (by experiment id) response]: {response}") | ||
| except Exception as error: | ||
| log_error("Send Reward (by experiment id)", error) | ||
|
|
||
|
|
||
| async def do_send_reward_by_decision_point(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.send_reward( | ||
| reward_value=reward_value, | ||
| context=context, | ||
| decision_point={"site": site, "target": target}, | ||
| ) | ||
| print(f"\n[Send Reward (by decision point) response]: {response}") | ||
| except Exception as error: | ||
| log_error("Send Reward (by decision point)", error) | ||
|
|
||
|
|
||
| async def do_log(client: UpgradeClient) -> None: | ||
| try: | ||
| response = await client.log(log_inputs) | ||
| print(f"\n[Log response]: {response}") | ||
| except Exception as error: | ||
| log_error("Log", error) | ||
|
|
||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Utility | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
|
|
||
| def log_error(function_context: str, error: Exception) -> None: | ||
| if isinstance(error, UpgradeApiError): | ||
| print(f"\n[{function_context} error]: HTTP {error.status_code} — {error.response_body}") | ||
| else: | ||
| print(f"\n[{function_context} error]: {error}") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| print(f"\n[quickTest] user_id={user_id} host={host_url} context={context}\n") | ||
| asyncio.run(quick_test()) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we add
from __future__ import annotationsafter the module docstring? The package supports Python 3.9+, but evaluatingstr | Nonehere raises aTypeErroron Python 3.9 when the function is defined, before it can even be called. Postponing annotation evaluation should address that compatibility issue.