alex/discord-aidolls
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Switch Model Command (#126)

Browse Source
This commit is contained in:
Kevin Dang
2024-10-12 22:03:31 -07:00
committed by GitHub
parent 5d02800c3f
commit 9f61f6bc6c
22 changed files with 334 additions and 376 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/setup-local.md
View File

@@ -11,14 +11,17 @@
* You can now interact with the model you just ran (it might take a second to startup).
* Response time varies with processing power!
> [!NOTE]
> You can now pull models directly from the Discord client using `/pull-model <model-name>` or `/switch-model <model-name>`. They must exist from your local model library or from the [Ollama Model Library](https://ollama.com/library)
## To Run Locally (without Docker)
* Run `npm install` to install the npm packages.
* Ensure that your [.env](../.env.sample) file's `OLLAMA_IP` is `127.0.0.1` to work properly.
* You only need your `CLIENT_TOKEN`, `MODEL`, `OLLAMA_IP`, `OLLAMA_PORT`.
* You only need your `CLIENT_TOKEN`, `OLLAMA_IP`, `OLLAMA_PORT`.
* The ollama ip and port should just use it's defaults by nature. If not, utilize `OLLAMA_IP = 127.0.0.1` and `OLLAMA_PORT = 11434`.
* Now, you can run the bot by running `npm run client` which will build and run the decompiled typescript and run the setup for ollama.
* **IMPORTANT**: This must be ran in the wsl/Linux instance to work properly! Using Command Prompt/Powershell/Git Bash/etc. will not work on Windows (at least in my experience).
* Refer to the [resources](../README.md#resources) on what node version to use.
* If you are using wsl, open up a separate terminal/shell to startup the ollama service. Again, if you are running an older ollama, you must run `ollama serve` in that shell.
* If you are on an actual Linux machine/VM there is no need for another terminal (unless you have an older ollama version).
* If you do not have a model, you will need to run `ollama pull [model name]` in a separate terminal to get it.
* If you do not have a model, you **can optionally** run `ollama pull [model name]` in wsl prior to application start. You are not required as it can be pulled from the Discord client.