alex/discord-ollama
1
0
Fork 0
mirror of https://github.com/kevinthedang/discord-ollama.git synced 2025-12-12 11:56:06 -05:00
Code Issues Packages Projects Releases Wiki Activity

User Preferences and Setup Docs (#20)

Browse Source 
* added message style command

* docker setup scripts

* reformat messageStyle.ts

* fix: register unregister on deploy

* add: messageStream preference

* add: json config handler

* update: messageCreate gets config

* update: shifted chat to config callback

* fix: naming conventions based on discord

* update: setup in docs now

* add: static docker ips

* version increment

* add: bot message for no config

* fix: no config case

* add: clarification for subnetting

* update: version increment in lock file

---------

Co-authored-by: JT2M0L3Y <jtsmoley@icloud.com>
This commit is contained in:
Kevin Dang
2024-03-22 10:37:06 -07:00
committed by GitHub
parent 5e74736c57
commit 43fb2ea94e
20 changed files with 377 additions and 146 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
docs/setup-docker.md Normal file
View File

@@ -0,0 +1,37 @@
## Docker Setup
* Follow this guide to setup [Docker](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04)
* If on Windows, download [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) to get the docker engine.
* Please also install [Docker Compose](https://docs.docker.com/compose/install/linux/) for easy running. If not, there are [scripts](#manual-run-with-docker) to set everything up.
## To Run (with Docker and Docker Compose)
* With the inclusion of subnets in the `docker-compose.yml`, you will need to set the `SUBNET_ADDRESS`, `OLLAMA_IP`, `OLLAMA_PORT`, and `DISCORD_IP`. Here are some default values if you don't care:
* `OLLAMA_IP = 172.18.0.2`
* `OLLAMA_PORT = 11434`
* `DISCORD_IP = 172.18.0.3`
* `SUBNET_ADDRESS = 172.18.0.0`
* Don't understand any of this? watch a Networking video to understand subnetting.
* You will need a model in the container for this to work properly, on Docker Desktop go to the `Containers` tab, select the `ollama` container, and select `Exec` to run as root on your container. Now, run `ollama pull [model name]` to get your model.
* For Linux Servers, you need another shell to pull the model, or if you run `docker compose build && docker compose up -d`, then it will run in the background to keep your shell. Run `docker exec -it ollama bash` to get into the container and run the samme pull command above.
* Otherwise, there is no need to install any npm packages for this, you just need to run `npm run start` to pull the containers and spin them up.
* For cleaning up on Linux (or Windows), run the following commands:
* `docker compose stop`
* `docker compose rm`
* `docker ps` to check if containers have been removed.
* You can also use `npm run clean` to clean up the containers and remove the network to address a possible `Address already in use` problem.
## Manual Run (with Docker)
* Run the following commands:
* `npm run docker:build`
* `npm run docker:ollama`
* `npm run docker:client`
* `docker ps` to see if the containers are there!
* Names should be **discord** and **ollama**.
* You can also just run `npm run docker:start` now for the above commands.
* Clean-up:
* `docker ps` for the conatiner id's. Use `-a` flag as necessary.
* `docker rm -f discord && docker rm -f ollama` to remove the containers.
* `docker rm -f CONTAINER_ID` do for both containers if naming issues arise.
* `docker network rm ollama-net` removes the network.
* `docker network prune` will also work so long as the network is unused.
* Remove Image:
* If you need to remove the image run `docker image rm IMAGE_ID`. You can get the image id by running `docker images`.

19
docs/setup-local.md Normal file
View File

@@ -0,0 +1,19 @@
## Ollama Setup
* Go to Ollama's [Linux download page](https://ollama.ai/download/linux) and run the simple curl command they provide. The command should be `curl https://ollama.ai/install.sh | sh`.
* Now the the following commands in separate terminals to test out how it works!
* In terminal 1 -> `ollama serve` to setup ollama
* In terminal 2 -> `ollama run [model name]`, for example `ollama run llama2`
* The models can vary as you can create your own model. You can also view ollama's [library](https://ollama.ai/library) of models.
* If there are any issues running ollama because of missing LLMs, run `ollama pull [model name]` as it will pull the model if Ollama has it in their library.
* This can also be done in [wsl](https://learn.microsoft.com/en-us/windows/wsl/install) for Windows machines.
* You can now interact with the model you just ran (it might take a second to startup).
* Response time varies with processing power!
## To Run Locally (without Docker)
* Run `npm install` to install the npm packages.
* Ensure that your [.env](../.env.sample) file's `OLLAMA_IP` is `127.0.0.1` to work properly.
* Now, you can run the bot by running `npm run client` which will build and run the decompiled typescript and run the setup for ollama.
* **IMPORTANT**: This must be ran in the wsl/Linux instance to work properly! Using Command Prompt/Powershell/Git Bash/etc. will not work on Windows (at least in my experience).
* Refer to the [resources](../README.md#resources) on what node version to use.
* Open up a separate terminal/shell (you will need wsl for this if on windows) and run `ollama serve` to startup ollama.
* If you do not have a model, you will need to run `ollama pull [model name]` in a separate terminal to get it.