1. Problems with Using LLMs Directly Unstable output One characteristic of generative AI is the diversity of its output. Ask an LLM the same question several times and you may get different answers. This uncertainty in output is a pleasant surprise for users in conversation and creative scenarios. But in

Mainly for standardizing the APIs I write myself, and also to help LLMs understand my APIs better. 1. Domain Use a dedicated domain whenever possible, for example api.example.com. 2. Path Always use lowercase letters. Do not include an /api prefix. Do not include file extensions. Do not put / at the end.

Because pricing and limits change over time, this article is only a reference for the time it was written. 1. Terminology RPM (requests per minute) Requests per minute RPD (requests per day) Requests per day TPM (tokens per minute) Tokens per minute TPD (tokens per day), Tokens per day

1. What Is an Ops Tool https://www.chenshaowen.com/ops/ is one of the tools I use most often in day-to-day operations. To operate on machines, I can reuse my previous scripts and perform operations in bulk. To operate on clusters, I can reuse my previous scripts and operate machines without logging into the nodes.

1. Running Locally in Containers Start the LLM 1 docker run --security-opt apparmor=unconfined --security-opt seccomp=unconfined --rm -p 8000:8000 shaowenchen/chinese-alpaca-2-7b-gguf:Q2_K The API documentation is available at http://localhost:8000/docs, as shown below: Deploy a simple Chat UI Note that the OPENAI_API_HOST parameter must be set to your host machine’s IP address rather than localhost or 127.