New (Python) example: simple drawings (rectangle, ellipse, arrow)

[gvanrossum]

For me, this works best with openai model gpt-3.5-turbo; gpt-4 seems worse (?!).

Here's a sample session (no longer valid without history support)

PATH=/opt/homebrew/opt/ccache/libexec:/opt/homebrew/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Users/guido/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/bash:/Users/guido/TypeChat/python/.venv/bin:/Library/Frameworks/Python.framework/Versions/3.13/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/Users/guido/.nvm/versions/node/v19.8.1/bin:/opt/homebrew/opt/ccache/libexec:/opt/homebrew/bin

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
(.venv) ~/TypeChat/python$ PYTHONPATH=src python examples/drawing/demo.py 
~> draw three squares of side 50 in a diagonal
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw three squares of side 50 in a diagonal\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
ReadTimeout('') raised from within internal TypeChat language model.
~> draw two squares of side 50 in a diagonal
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    }
  ]
}
~> make it three squares
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

retrying (ReadTimeout('')) ...
retrying (ReadTimeout('')) ...
{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    },
    {
      "type": "Box",
      "x": 100,
      "y": 100,
      "width": 50,
      "height": 50,
      "text": "",
      "style": null
    }
  ]
}
~> label them according to size
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 100,\n      "y": 100,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nlabel them according to size\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "Small",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "Medium",
      "style": null
    },
    {
      "type": "Box",
      "x": 100,
      "y": 100,
      "width": 50,
      "height": 50,
      "text": "Large",
      "style": null
    }
  ]
}
~> oops, label them according to position
--------- NEXT REQUEST ---------
{'role': 'user', 'content': '\nYou are a service that translates user requests into JSON objects of type "Drawing" according to the following TypeScript definitions:\n```\n// A drawing is a list of boxes. (We\'ll add other elements later, like arrows.)\ninterface Drawing {\n    items: Array<Box | UnknownText>;\n}\n\n// Use this type for input that match nothing else\ninterface UnknownText {\n    type: "Unknown";\n    // The text that wasn\'t understood\n    text: string;\n}\n\n// A rectangular box.\n// \n// The coordinate system has origin top left, x points right, y points down.\n// Measurements are in pixels.\n// \n// There can also be text in the box. There are optional style properties.\ninterface Box {\n    type: "Box";\n    // Top left corner coordinates\n    x: number;\n    y: number;\n    // Size of the box\n    width: number;\n    height: number;\n    // Text centered in the box\n    text: string;\n    // Box drawing style (optional)\n    style: Style | null;\n}\n\ninterface Style {\n    type: "Style";\n    corners: "rounded" | "sharp";\n}\n\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\ndraw two squares of side 50 in a diagonal\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nmake it three squares\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 100,\n      "y": 100,\n      "width": 50,\n      "height": 50,\n      "text": "",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\nlabel them according to size\n'''\n"}
{'role': 'assistant', 'content': '\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n```\n{\n  "items": [\n    {\n      "type": "Box",\n      "x": 0,\n      "y": 0,\n      "width": 50,\n      "height": 50,\n      "text": "Small",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 50,\n      "y": 50,\n      "width": 50,\n      "height": 50,\n      "text": "Medium",\n      "style": null\n    },\n    {\n      "type": "Box",\n      "x": 100,\n      "y": 100,\n      "width": 50,\n      "height": 50,\n      "text": "Large",\n      "style": null\n    }\n  ]\n}\n```\n'}
{'role': 'user', 'content': "\nThe following is a user request:\n'''\noops, label them according to position\n'''\n\nThe following is the user request translated into a JSON object with 2 spaces of indentation and no properties with the value undefined:\n"}

retrying (ReadTimeout('')) ...
{
  "items": [
    {
      "type": "Box",
      "x": 0,
      "y": 0,
      "width": 50,
      "height": 50,
      "text": "First",
      "style": null
    },
    {
      "type": "Box",
      "x": 50,
      "y": 50,
      "width": 50,
      "height": 50,
      "text": "Second",
      "style": null
    },
    {
      "type": "Box",
      "x": 100,
      "y": 100,
      "width": 50,
      "height": 50,
      "text": "Third",
      "style": null
    }
  ]
}
~> 

[DanielRosenwasser]

There's some stuff here that makes intuitive sense, but I'll need to check in with others (CC @ahejlsberg)

I would think that the best format for incorporating chat history would be something like

• System: The assistant is a bot that responds in JSON according to a schema.
• ...: In-between messages alternating between user/assistant
• User: Some request
• System: Translate the prior request with JSON that satisfies Type in the following schema:

That gives background for the current convo, plus reinforces the current task at hand. This PR is pretty close to that.

[gvanrossum]

I wonder how to proceed. Clearly this PR two things (adding history and adding a new example) and that's not great. Your fixes to the translator (#240) broke my PR and I would rather not try to merge it back. I propose to add my example without chat history and start a separate discussion on chat history -- okay?

[Copilot]

Pull request overview

Adds a new Python “drawing” example that translates natural-language requests into a typed drawing schema and renders the result with Tkinter, plus a couple of small usability/dependency updates elsewhere.

Changes:

• Add python/examples/drawing example (schema + Tkinter renderer + interactive demo + sample input).
• Make readline usage in the interactive request loop optional (avoid crashing where it’s unavailable).
• Update the CoffeeShop notebook’s bootstrap installs (add python-dotenv and tabulate).

Show a summary per file

File	Description
python/src/typechat/_internal/interactive.py	Makes readline import optional for interactive mode.
python/notebooks/coffeeShop.ipynb	Adds notebook runtime deps (python-dotenv, tabulate).
python/examples/drawing/schema.py	Introduces a drawing schema (Box/Ellipse/Arrow/etc.) with style metadata.
python/examples/drawing/render.py	Adds Tkinter rendering for the drawing schema.
python/examples/drawing/demo.py	Adds an interactive demo that maintains simple request history and re-renders.
python/examples/drawing/input.txt	Provides sample prompts for the drawing demo.
python/examples/drawing/init.py	Initializes the new example package.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 7/7 changed files
• Comments generated: 5

[robgruen]

@copilot resolve the merge conflicts in this pull request