Compare commits
10 Commits
e2bd7ed4c6
...
957f8bb8b3
Author | SHA1 | Date | |
---|---|---|---|
![]() |
957f8bb8b3 | ||
![]() |
36ae2f2021 | ||
![]() |
8f78a4621f | ||
![]() |
936f4bb418 | ||
![]() |
8433e93ff6 | ||
![]() |
7495a4f0a1 | ||
![]() |
77178347ea | ||
![]() |
f758e62fa3 | ||
![]() |
31cb3ac354 | ||
![]() |
3494ce23f9 |
@@ -8,6 +8,8 @@ author = "Carl Pearson"
|
|||||||
|
|
||||||
## Community
|
## Community
|
||||||
|
|
||||||
|
- IPDPS'24 AsHES workshop program chair
|
||||||
|
- SC'23 ExaMPI workshop committee member
|
||||||
- ICPP'23 program committee member
|
- ICPP'23 program committee member
|
||||||
- ISC'23 workshop commitee member
|
- ISC'23 workshop commitee member
|
||||||
- IPDPS'23 AsHES Workshop program committee member
|
- IPDPS'23 AsHES Workshop program committee member
|
||||||
|
@@ -1,7 +1,7 @@
|
|||||||
+++
|
+++
|
||||||
title = "Interesting Links"
|
title = "Interesting Links"
|
||||||
date = 2022-11-26T00:00:00
|
date = 2022-11-26T00:00:00
|
||||||
lastmod = 2022-11-27T00:00:00
|
lastmod = 2023-06-04T00:00:00
|
||||||
draft = false
|
draft = false
|
||||||
|
|
||||||
# Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`.
|
# Authors. Comma separated list, e.g. `["Bob Smith", "David Jones"]`.
|
||||||
@@ -58,6 +58,22 @@ Presented without comment or endorsement.
|
|||||||
* [A Chemical Hunger – Interlude I: The Fattest Cities in the Land](https://slimemoldtimemold.com/2021/11/08/a-chemical-hunger-interlude-i-the-fattest-cities-in-the-land/)
|
* [A Chemical Hunger – Interlude I: The Fattest Cities in the Land](https://slimemoldtimemold.com/2021/11/08/a-chemical-hunger-interlude-i-the-fattest-cities-in-the-land/)
|
||||||
* [A Chemical Hunger – Part X: What to Do About It](https://slimemoldtimemold.com/2021/11/23/a-chemical-hunger-part-x-what-to-do-about-it/)-->
|
* [A Chemical Hunger – Part X: What to Do About It](https://slimemoldtimemold.com/2021/11/23/a-chemical-hunger-part-x-what-to-do-about-it/)-->
|
||||||
|
|
||||||
|
## June 2023
|
||||||
|
|
||||||
|
* [DreamBerd Programming Language](https://github.com/TodePond/DreamBerd)
|
||||||
|
|
||||||
|
## May 2023
|
||||||
|
|
||||||
|
* [Sandspiel](https://sandspiel.club/#)
|
||||||
|
* [DALLE mini](https://craiyon.com)
|
||||||
|
* [The Scaling Hyopthesis](https://gwern.net/scaling-hypothesis)
|
||||||
|
* [Learning the Elite Class](https://aella.substack.com/p/learning-the-elite-class)
|
||||||
|
* [I'm trying to put a command in a variable, but the complex cases always fail!](http://mywiki.wooledge.org/Bash/FAQ/050)
|
||||||
|
* [(manifold) Will and independent RCT study evaluating Silexan for anxiety find a standardized effect size greater than 0.5?](https://manifold.markets/AlvarodeMenard/will-an-independent-rct-study-evalu)
|
||||||
|
* [Things your manager might not know](jvns.ca/blog/things-your-manager-might-not-know/)
|
||||||
|
* [doodle : ai](https://doodleai.darraghoriordan.com/)
|
||||||
|
|
||||||
|
|
||||||
## November 2022
|
## November 2022
|
||||||
|
|
||||||
* [RFC 1855: Netiquette Guidelines](https://www.rfc-editor.org/rfc/rfc1855) ([archive](https://web.archive.org/web/20221118154752/https://www.rfc-editor.org/rfc/rfc1855))
|
* [RFC 1855: Netiquette Guidelines](https://www.rfc-editor.org/rfc/rfc1855) ([archive](https://web.archive.org/web/20221118154752/https://www.rfc-editor.org/rfc/rfc1855))
|
||||||
|
@@ -1,5 +1,5 @@
|
|||||||
+++
|
+++
|
||||||
title = "Running Vicuna-13B in the Cloud"
|
title = "Running Vicuna-13B in Google Cloud"
|
||||||
date = 2023-05-06T00:00:00
|
date = 2023-05-06T00:00:00
|
||||||
lastmod = 2023-05-06T00:00:00
|
lastmod = 2023-05-06T00:00:00
|
||||||
draft = false
|
draft = false
|
||||||
@@ -9,7 +9,7 @@ authors = ["Carl Pearson"]
|
|||||||
|
|
||||||
tags = []
|
tags = []
|
||||||
|
|
||||||
summary = ""
|
summary = "How to experiment with hosting Vicuna-13B on a cloud VM"
|
||||||
|
|
||||||
# Projects (optional).
|
# Projects (optional).
|
||||||
# Associate this post with one or more of your projects.
|
# Associate this post with one or more of your projects.
|
||||||
@@ -39,7 +39,11 @@ categories = []
|
|||||||
|
|
||||||
+++
|
+++
|
||||||
|
|
||||||
Vicuna-13B is an LLM based off of the LLaMa model.
|
[Vicuna-13B](https://lmsys.org/blog/2023-03-30-vicuna/) is an LLM chatbot based off of the LLaMa model.
|
||||||
|
It's authors claim it achieves 90% of the quality of ChatGPT in a "fun and non-scientific" evaluation.
|
||||||
|
|
||||||
|
You can rent some cloud hardware and experiment with Vicuna 13B yourself!
|
||||||
|
Using CPU-only is slow (couple tokens per second), but fast enough for yout to get an idea of what to expect.
|
||||||
|
|
||||||
## Set up your Cloud Instance
|
## Set up your Cloud Instance
|
||||||
|
|
||||||
@@ -47,7 +51,9 @@ Create a cloud VM with
|
|||||||
* 150 GB of disk space
|
* 150 GB of disk space
|
||||||
* 64 GB of CPU memory
|
* 64 GB of CPU memory
|
||||||
|
|
||||||
When everything was done, my VM had 132GB of disk space used, so you'll probably want at least 150GB
|
I used a Google Compute Engine `e2-standard-16`, which costs around $0.70/hour, so it may not be something you want to leave running. You can stop the instance when you're not using it.
|
||||||
|
|
||||||
|
When everything was done, my VM had 132GB of disk space used.
|
||||||
|
|
||||||
Ordinarily I wouldn't recommend setting up python like this, but since we're just experimenting:
|
Ordinarily I wouldn't recommend setting up python like this, but since we're just experimenting:
|
||||||
|
|
||||||
@@ -123,7 +129,7 @@ I got an error about regenerating protobuf functions if I used protobuf > 3.20.
|
|||||||
|
|
||||||
## Apply the vicuna deltas
|
## Apply the vicuna deltas
|
||||||
|
|
||||||
FastChat has done the work of getting a little chat interface set up.
|
[FastChat](https://github.com/lm-sys/FastChat) has done the work of getting a little chat interface set up.
|
||||||
We'll use their package to download the deltas and apply them as well.
|
We'll use their package to download the deltas and apply them as well.
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -138,4 +144,8 @@ I had `fschat 0.2.5`.
|
|||||||
|
|
||||||
## Start Chatting
|
## Start Chatting
|
||||||
|
|
||||||
python3 -m fastchat.serve.cli --device cpu --model-path vicuna-13b/
|
This will open up a little Chat-GPT-style interface in your terminal.
|
||||||
|
|
||||||
|
```
|
||||||
|
python3 -m fastchat.serve.cli --device cpu --model-path vicuna-13b/
|
||||||
|
```
|
Reference in New Issue
Block a user