Draft
Conversation
* add init to panza to turn it into a package * add pyproject.toml but no dependencies yet * add the rest of the panzamail dependencies :) * install dependencies based on pyproject.toml instead of raw pip and conda commands
* Add Ollama inference * expose Panza as a web server * add api keys to env variables and check in server * check api key * switch to fastapi to prevent model reloading * Add ollama-backed streaming HTTP server --------- Co-authored-by: Armand Nicolicioiu <armand.nicolicioiu@gmail.com>
…of manual modifications
- Added bug fix for error encountered in json dumps for Message and mboxMessage objects - Added clarification for email and username reqs - Changed wanbd_disabled default to true to track with README
maddox-j
commented
Nov 6, 2024
Contributor
Author
There was a problem hiding this comment.
Need to rename, and to link back to the original README
maddox-j
commented
Nov 6, 2024
|
|
||
| If running with Ollama, then Ollama needs to be installed from the [web page](https://ollama.com/). | ||
|
|
||
| Then, you will need to convert your model into a GGUF file. |
Contributor
Author
There was a problem hiding this comment.
is it beneficial to add more support for this?
maddox-j
commented
Nov 6, 2024
|
|
||
| - To run Panza after a full training run, try something like `CUDA_VISIBLE_DEVICES=0 python3 runner.py user=USERNAME interfaces=cli writer/llm=transformers`. | ||
| - To run Panza after a RoSA or LoRA training run, replace `writer/llm=transformers` with `writer/llm=peft` TODO Armand: can we fix this? | ||
|
|
Contributor
Author
There was a problem hiding this comment.
Integrate with the inference markdown+ resolve TODO
maddox-j
commented
Nov 6, 2024
| @@ -0,0 +1,9 @@ | |||
| email_address: "abc@xyz.com" # Change this to your email address! | |||
| username: "abc" # TODO(armand): Use custom resolver to extract username from email address. | |||
|
|
|||
maddox-j
commented
Nov 6, 2024
| </div> | ||
|
|
||
|
|
||
| ## TODO: Prerequisites |
This commit features a series of updates. 1. Introduction of formatting with Black added through a precommit that contributers should install. Instructions to do so have been added so that if PRs are created, all code is in same formatting. 2. Formatting code with Black. 3. Removal of debug print statements. 4. Addressing bug with n_proc > in datasets.map with HF
…nto jen/eval-refactor
…nto jen/eval-refactor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding a WIP PR to manage the codebase refactor merge to main