From 1c62958c9f5be412109199a5465fb6582067831c Mon Sep 17 00:00:00 2001 From: Kevin Dang Date: Wed, 7 Feb 2024 10:49:42 -0800 Subject: [PATCH] docker setup instructions --- README.md | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 16a021d..788d95e 100644 --- a/README.md +++ b/README.md @@ -11,27 +11,42 @@ Ollama is an AI model management tool that allows users to install and use custo * You can now interact with the model you just ran (it might take a second to startup). * Response time varies with processing power! -## To Run +## Project Setup * Clone this repo using `git clone https://github.com/kevinthedang/discord-ollama.git` or just use [GitHub Desktop](https://desktop.github.com/) to clone the repo. -* Run `npm install` to install the npm packages. -* You will need a `.env` file in the root of the project directory with the bot's token. +* You will need a `.env` file in the root of the project directory with the bot's token. There is a `.env.sample` is provided for you as a reference for what environment variables. * For example, `CLIENT_TOKEN = [Bot Token]` -* Now, you can run the bot by running `npm run start` which will build and run the decompiled typescript and run the setup for ollama. + +## To Run (with Docker) +* Follow this guide to setup [Docker](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04) + * If on Windows, download [Docker Desktop](https://docs.docker.com/desktop/install/windows-install/) to get the docker engine. +* You will need a model in the container for this to work properly, on Docker Desktop go to the `Containers` tab, select the `ollama` container, and select `Exec` to run as root on your container. Now, run `ollama pull [model name]` to get your model. + * For Linux Servers, you need another shell to pull the model, or if you run `docker-compose build && docker-compose up -d`, then it will run in the background to keep your shell. Run `docker exec -it ollama bash` to get into the container and run the samme pull command above. +* There is no need to install any npm packages for this, you just need to run `npm run start` to pull the containers and spin them up. +* For cleaning up on Linux (or Windows), run the following commands: + * `docker-compose stop` + * `docker-compose rm` + * `docker ps` to check if containers have been removed. + +## To Run Locally (without Docker) +* Run `npm install` to install the npm packages. +* Now, you can run the bot by running `npm run client` which will build and run the decompiled typescript and run the setup for ollama. * **IMPORTANT**: This must be ran in the wsl/Linux instance to work properly! Using Command Prompt/Powershell/Git Bash/etc. will not work on Windows (at least in my experience). * Refer to the [resources](#resources) on what node version to use. +* Open up a separate terminal/shell (you will need wsl for this if on windows) and run `ollama serve` to startup ollama. + * If you do not have a model, you will need to run `ollama pull [model name]` in a separate terminal to get it. ## Resources * [NodeJS](https://nodejs.org/en) - * This project uses `v20.10.0` (npm `10.2.5`). Consider using [nvm](https://github.com/nvm-sh/nvm) for multiple NodeJS versions. + * This project uses `v20.10.0+` (npm `10.2.5`). Consider using [nvm](https://github.com/nvm-sh/nvm) for multiple NodeJS versions. * To run dev in `ts-node`, using `v18.18.2` is recommended. **CAUTION**: `v18.19.0` or `lts/hydrogen` will not run properly. * To run dev with `tsx`, you can use `v20.10.0` or earlier. * This project supports any NodeJS version above `16.x.x` to only allow ESModules. * [Ollama](https://ollama.ai/) * [Ollama Docker Image](https://hub.docker.com/r/ollama/ollama) - * **IMPORTANT**: For Nvidia GPU setup, **install** `nvidia container toolkit` then **configure** it with Docker to utilize Nvidia driver. + * **IMPORTANT**: For Nvidia GPU setup, **install** `nvidia container toolkit/runtime` then **configure** it with Docker to utilize Nvidia driver. * [Discord Developer Portal](https://discord.com/developers/docs/intro) * [Discord.js Docs](https://discord.js.org/docs/packages/discord.js/main) -* [Setting up Docker (Ubuntu)][https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04] +* [Setting up Docker (Ubuntu 20.04)](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04) * [Setting up Nvidia Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) ## Acknowledgement