Compare commits

..

10 Commits

Author SHA1 Message Date
Carl Pearson
957f8bb8b3 . 2023-09-06 14:27:29 -06:00
Carl Pearson
36ae2f2021 AsHES'24 program chair 2023-09-06 14:19:54 -06:00
Carl Pearson
8f78a4621f more links 2023-06-04 12:56:19 -06:00
Carl Pearson
936f4bb418 some more links 2023-06-04 06:21:14 -06:00
Carl Pearson
8433e93ff6 some more links 2023-06-04 06:15:31 -06:00
Carl Pearson
7495a4f0a1 SC'23 ExaMPI workshop committee member 2023-05-08 13:05:01 -06:00
Carl Pearson
77178347ea a Vicuna-13B link 2023-05-08 13:01:33 -06:00
Carl Pearson
f758e62fa3 a bit more Vicuna context 2023-05-08 13:01:06 -06:00
Carl Pearson
31cb3ac354 Vicuna updates 2023-05-08 06:03:34 -06:00
Carl Pearson
3494ce23f9 vicuna updates 2023-05-08 06:02:06 -06:00
3 changed files with 35 additions and 7 deletions

View File

@@ -8,6 +8,8 @@ author = "Carl Pearson"
## Community ## Community
- IPDPS'24 AsHES workshop program chair
- SC'23 ExaMPI workshop committee member
- ICPP'23 program committee member - ICPP'23 program committee member
- ISC'23 workshop commitee member - ISC'23 workshop commitee member
- IPDPS'23 AsHES Workshop program committee member - IPDPS'23 AsHES Workshop program committee member

View File

@@ -1,7 +1,7 @@
+++ +++
title = "Interesting Links" title = "Interesting Links"
date = 2022-11-26T00:00:00 date = 2022-11-26T00:00:00
lastmod = 2022-11-27T00:00:00 lastmod = 2023-06-04T00:00:00
draft = false draft = false
# Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`. # Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`.
@@ -58,6 +58,22 @@ Presented without comment or endorsement.
* [A Chemical Hunger Interlude I: The Fattest Cities in the Land](https://slimemoldtimemold.com/2021/11/08/a-chemical-hunger-interlude-i-the-fattest-cities-in-the-land/) * [A Chemical Hunger Interlude I: The Fattest Cities in the Land](https://slimemoldtimemold.com/2021/11/08/a-chemical-hunger-interlude-i-the-fattest-cities-in-the-land/)
* [A Chemical Hunger Part X: What to Do About It](https://slimemoldtimemold.com/2021/11/23/a-chemical-hunger-part-x-what-to-do-about-it/)--> * [A Chemical Hunger Part X: What to Do About It](https://slimemoldtimemold.com/2021/11/23/a-chemical-hunger-part-x-what-to-do-about-it/)-->
## June 2023
* [DreamBerd Programming Language](https://github.com/TodePond/DreamBerd)
## May 2023
* [Sandspiel](https://sandspiel.club/#)
* [DALLE mini](https://craiyon.com)
* [The Scaling Hyopthesis](https://gwern.net/scaling-hypothesis)
* [Learning the Elite Class](https://aella.substack.com/p/learning-the-elite-class)
* [I'm trying to put a command in a variable, but the complex cases always fail!](http://mywiki.wooledge.org/Bash/FAQ/050)
* [(manifold) Will and independent RCT study evaluating Silexan for anxiety find a standardized effect size greater than 0.5?](https://manifold.markets/AlvarodeMenard/will-an-independent-rct-study-evalu)
* [Things your manager might not know](jvns.ca/blog/things-your-manager-might-not-know/)
* [doodle : ai](https://doodleai.darraghoriordan.com/)
## November 2022 ## November 2022
* [RFC 1855: Netiquette Guidelines](https://www.rfc-editor.org/rfc/rfc1855) ([archive](https://web.archive.org/web/20221118154752/https://www.rfc-editor.org/rfc/rfc1855)) * [RFC 1855: Netiquette Guidelines](https://www.rfc-editor.org/rfc/rfc1855) ([archive](https://web.archive.org/web/20221118154752/https://www.rfc-editor.org/rfc/rfc1855))

View File

@@ -1,5 +1,5 @@
+++ +++
title = "Running Vicuna-13B in the Cloud" title = "Running Vicuna-13B in Google Cloud"
date = 2023-05-06T00:00:00 date = 2023-05-06T00:00:00
lastmod = 2023-05-06T00:00:00 lastmod = 2023-05-06T00:00:00
draft = false draft = false
@@ -9,7 +9,7 @@ authors = ["Carl Pearson"]
tags = [] tags = []
summary = "" summary = "How to experiment with hosting Vicuna-13B on a cloud VM"
# Projects (optional). # Projects (optional).
# Associate this post with one or more of your projects. # Associate this post with one or more of your projects.
@@ -39,7 +39,11 @@ categories = []
+++ +++
Vicuna-13B is an LLM based off of the LLaMa model. [Vicuna-13B](https://lmsys.org/blog/2023-03-30-vicuna/) is an LLM chatbot based off of the LLaMa model.
It's authors claim it achieves 90% of the quality of ChatGPT in a "fun and non-scientific" evaluation.
You can rent some cloud hardware and experiment with Vicuna 13B yourself!
Using CPU-only is slow (couple tokens per second), but fast enough for yout to get an idea of what to expect.
## Set up your Cloud Instance ## Set up your Cloud Instance
@@ -47,7 +51,9 @@ Create a cloud VM with
* 150 GB of disk space * 150 GB of disk space
* 64 GB of CPU memory * 64 GB of CPU memory
When everything was done, my VM had 132GB of disk space used, so you'll probably want at least 150GB I used a Google Compute Engine `e2-standard-16`, which costs around $0.70/hour, so it may not be something you want to leave running. You can stop the instance when you're not using it.
When everything was done, my VM had 132GB of disk space used.
Ordinarily I wouldn't recommend setting up python like this, but since we're just experimenting: Ordinarily I wouldn't recommend setting up python like this, but since we're just experimenting:
@@ -123,7 +129,7 @@ I got an error about regenerating protobuf functions if I used protobuf > 3.20.
## Apply the vicuna deltas ## Apply the vicuna deltas
FastChat has done the work of getting a little chat interface set up. [FastChat](https://github.com/lm-sys/FastChat) has done the work of getting a little chat interface set up.
We'll use their package to download the deltas and apply them as well. We'll use their package to download the deltas and apply them as well.
``` ```
@@ -138,4 +144,8 @@ I had `fschat 0.2.5`.
## Start Chatting ## Start Chatting
python3 -m fastchat.serve.cli --device cpu --model-path vicuna-13b/ This will open up a little Chat-GPT-style interface in your terminal.
```
python3 -m fastchat.serve.cli --device cpu --model-path vicuna-13b/
```