Stanford的这篇论文及后续的AI小镇出来后，我非常感兴趣，想尝试着复现，然后在云服务器上运行做一个自我运作的游戏，考虑到chatgpt的调用费用，我使用能在CPU上推理的大模型来做替代，以下是我在复现时遇到的一些问题总结。
相关论文
A Survey on Large Language Model based Autonomous Agents
ExpeL: LLM Agents Are Experiential Learners
Generative Agents: Interactive Simulacra of Human Behavior arxiv
代码及讨论
generative agents github
a16z 使用游戏框架convex重写 github
rlancemartin 使用本地跑llama2来做agent github
讨论：https://news.ycombinator.com/item?id=37128293
git clone慢，在原地址前加https://ghproxy.com
git clone https://ghproxy.com/https://github.com/tensorflow/tensorflow
# git clone https://ghproxy.com/<git地址> 注意：git地址前无需加空格大模型获取
Chinese-LLAMA-Alpaca-2 github
Ollama 获取日志如下
$ ollama pull llama2:7b-chat-q8_0 # 获取大模型
pulling 3bfdde943555... 100% |████████████████████████| (7.2/7.2 GB, 5.6 MB/s)
pulling 8c17c2ebb0ea... 100% |█████████████████████████████| (7.0/7.0 kB, 36 MB/s)
pulling 7c23fb36d801... 100% |████████████████████████████| (4.8/4.8 kB, 9.8 MB/s)
pulling 578a2e81f706... 100% |███████████████████████████████| (95/95 B, 109 kB/s)
pulling e35ab70a78c7... 100% |███████████████████████████████| (90/90 B, 775 kB/s)
pulling 79918675e4d5... 100% |█████████████████████████████| (447/447 B, 3.2 MB/s)
verifying sha256 digest
writing manifest
successMeta Facebook 获取日志如下
$ cd ~/Documents/CodeSpace/llama
Enter the URL from email: https://download.llamameta.net/*?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
Enter the list of models to download without spaces (7B,13B,70B,7B-chat,13B-chat,70B-chat), or press Enter for all: 7B
Downloading LICENSE and Acceptable Usage Policy
--2023-08-19 08:46:25-- https://download.llamameta.net/LICENSE?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.23, 18.154.144.45, 18.154.144.56, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.23|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：7020 (6.9K) [binary/octet-stream]
正在保存至: “./LICENSE”
./LICENSE 100%[===========================>] 6.86K --.-KB/s 用时 0.002s
2023-08-19 08:46:26 (3.16 MB/s) - 已保存 “./LICENSE” [7020/7020])
--2023-08-19 08:46:26-- https://download.llamameta.net/USE_POLICY.md?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.45, 18.154.144.56, 18.154.144.95, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.45|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：4766 (4.7K) [binary/octet-stream]
正在保存至: “./USE_POLICY.md”
./USE_POLICY.md 100%[===========================>] 4.65K --.-KB/s 用时 0s
2023-08-19 08:46:27 (60.6 MB/s) - 已保存 “./USE_POLICY.md” [4766/4766])
Downloading tokenizer
--2023-08-19 08:46:27-- https://download.llamameta.net/tokenizer.model?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.56, 18.154.144.95, 18.154.144.23, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.56|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：499723 (488K) [binary/octet-stream]
正在保存至: “./tokenizer.model”
./tokenizer.model 100%[===========================>] 488.01K 859KB/s 用时 0.6s
2023-08-19 08:46:28 (859 KB/s) - 已保存 “./tokenizer.model” [499723/499723])
--2023-08-19 08:46:28-- https://download.llamameta.net/tokenizer_checklist.chk?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.95, 18.154.144.23, 18.154.144.45, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.95|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：50 [binary/octet-stream]
正在保存至: “./tokenizer_checklist.chk”
./tokenizer_checklist. 100%[===========================>] 50 --.-KB/s 用时 0s
2023-08-19 08:46:29 (1.40 MB/s) - 已保存 “./tokenizer_checklist.chk” [50/50])
download.sh: line 23: md5sum: command not found
Downloading llama-2-7b
--2023-08-19 08:46:29-- https://download.llamameta.net/llama-2-7b/consolidated.00.pth?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.23, 18.154.144.45, 18.154.144.56, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.23|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：13476925163 (13G) [binary/octet-stream]
正在保存至: “./llama-2-7b/consolidated.00.pth”
./llama-2-7b/consolida 100%[===========================>] 12.55G 19.8MB/s 用时 10m 4s
2023-08-19 08:56:33 (21.3 MB/s) - 已保存 “./llama-2-7b/consolidated.00.pth” [13476925163/13476925163])
--2023-08-19 08:56:34-- https://download.llamameta.net/llama-2-7b/params.json?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.95, 18.154.144.56, 18.154.144.45, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.95|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：102 [application/json]
正在保存至: “./llama-2-7b/params.json”
./llama-2-7b/params.js 100%[===========================>] 102 --.-KB/s 用时 0s
2023-08-19 08:56:35 (3.14 MB/s) - 已保存 “./llama-2-7b/params.json” [102/102])
--2023-08-19 08:56:35-- https://download.llamameta.net/llama-2-7b/checklist.chk?Policy=eyJTdGF0ZW1lbnQiOlt7InVuaXF1ZV9oYXNoIjoiOWVlbGN6a3RrbmN3Z2dud3A1b3ZuMTVpIiwiUmVzb3VyY2UiOiJodHRwczpcL1wvZG93bmxvYWQubGxhbWFtZXRhLm5ldFwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTY5MjQ1MzEyMn19fV19&Signature=b1hcpiSrxXv2bm8Wny16bVdMOlA74oFlRghK8zPoP%7E07A63sVGYnRQHSHS76XCDh-EJQ4Wov6LiQG2nAbxFQZzfTYgL8TLpeL82iVHryXlhFHACuY5CvdGLep53L1k8XUJvfhgDuKx5RMfmCsvBJ-GSfMkKQWW%7EI8qIrf15Mp6-huRkmv0pRKNR6e3dblrNc7yB6J2EzjppiMA5Fv4qhwAO0HlTOsiNd197aTxNYyuG9PkVafzdS0dxaFnMX%7EuqKke%7ErRF-YrYFYeQ9pgc3wF-epCjrENZBUZvFBIhgdTbFLYp0UKivsjuCoEok3goQJq17R9Hs3vXJoX1xHuBLNdg__&Key-Pair-Id=K15QRJLYKIFSLZ&Download-Request-ID=947735412959088
正在解析主机 download.llamameta.net (download.llamameta.net)… 18.154.144.56, 18.154.144.45, 18.154.144.23, …
正在连接 download.llamameta.net (download.llamameta.net)|18.154.144.56|:443… 已连接。
已发出 HTTP 请求，正在等待回应… 200 OK
长度：100 [binary/octet-stream]
正在保存至: “./llama-2-7b/checklist.chk”
./llama-2-7b/checklist 100%[===========================>] 100 --.-KB/s 用时 0s
2023-08-19 08:56:36 (8.67 MB/s) - 已保存 “./llama-2-7b/checklist.chk” [100/100])
Checking checksums
download.sh: line 58: md5sum: command not found模拟环境
按rlancemartin的配置搭建环境并运行
conda create -n llmagents python=3.9.12
pip install -r requirements.txt # 安装环境，使用pip3产生了numpy安装错误运行 python reverie.py ，出现如下报错
RuntimeError: no validator found for , see arbitrary_types_allowed in Config
根据 Error when importing ChatOutputParser: no validator found for #8158 将pydantic 1.9.2 更新至1.10.8
pip install --upgrade pydantic==1.10.8
但发现第一步都还没做，就先关闭后先做第一步
运行环境服务器
$ python manage.py runserver
/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/storages/init.py:9: UserWarning: This library has been designated as the official successor of django-storages and releases under that namespace. Please update your requirements files to point to django-storages.
warnings.warn('This library has been designated as the official successor of django-storages and '
/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/storages/init.py:9: UserWarning: This library has been designated as the official successor of django-storages and releases under that namespace. Please update your requirements files to point to django-storages.
warnings.warn('This library has been designated as the official successor of django-storages and '
Watching for file changes with StatReloader
Performing system checks…
System check identified no issues (0 silenced).
August 19, 2023 - 15:06:47
Django version 2.2, using settings 'frontend_server.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.运行模拟服务器
$ python reverie.py
Enter the name of the forked simulation: base_the_ville_isabella_maria_klaus
Enter the name of the new simulation: test-simulation
Note: The agents in this simulation package are computational
constructs powered by generative agents architecture and LLM. We
clarify that these agents lack human-like agency, consciousness,
and independent decision-making.
Enter option: run 2
Traceback (most recent call last):
File "/Users/yuanfang/Documents/CodeSpace/generative_agents/reverie/backend_server/reverie.py", line 471, in open_server
rs.start_server(int_count)
File "/Users/yuanfang/Documents/CodeSpace/generative_agents/reverie/backend_server/reverie.py", line 379, in start_server
next_tile, pronunciatio, description = persona.move(
File "/Users/yuanfang/Documents/CodeSpace/generative_agents/reverie/backend_server/persona/persona.py", line 220, in move
perceived = self.perceive(maze)
File "/Users/yuanfang/Documents/CodeSpace/generative_agents/reverie/backend_server/persona/persona.py", line 107, in perceive
return perceive(self, maze)
File "/Users/yuanfang/Documents/CodeSpace/generative_agents/reverie/backend_server/persona/cognitive_modules/perceive.py", line 144, in perceive
event_embedding = get_embedding(desc_embedding_in)
File "/Users/yuanfang/Documents/CodeSpace/generative_agents/reverie/backend_server/persona/prompt_template/gpt_structure.py", line 252, in get_embedding
return openai.Embedding.create(
File "/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/openai/api_resources/embedding.py", line 33, in create
response = super().create(*args, **kwargs)
File "/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 149, in create
) = cls.prepare_create_request( File "/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 106, in __prepare_create_request requestor = api_requestor.APIRequestor( File "/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/openai/api_requestor.py", line 130, in __init
self.api_key = key or util.default_api_key()
File "/Users/yuanfang/miniconda3/envs/llmagents/lib/python3.9/site-packages/openai/util.py", line 186, in default_api_key
raise openai.error.AuthenticationError(
openai.error.AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = ', or you can set the environment variable OPENAI_API_KEY=). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path = '. You can generate API keys in the OpenAI web interface. See https://onboard.openai.com for details, or email support@openai.com if you have any questions.
Error.
Enter option:这里出现文本Embedding调用错误，查了下，代码里通过调用OPENAI的Embedding服务，即text-embedding-ada-002模型（$0.0001 / 1K tokens pricing）来算文本的向量，以检索等操作。
如果要用OpenAI的服务的话，就得注册账号，但OpenAI又限制中国手机号，得租个外国手机号（见超详细注册OpenAI接口账号的教程）。考虑到OpenAI后期风控二次校验手机号的问题，我准备先自己跑个文本转向量的模型，在MTEB Leaderboard上挑了英文检索领域较好的bge-base-en，下载完成后在llmagents环境下由于缺少sentence_transformers而无法运行，安装又出现Failed to build safetensors的错误，不知道怎么解决，只好使用transformers的方式，但装着装着又显示pytorch不兼容，为了兼容，装着装着又把pytorch给删除了Orz
重新安装了下，也会出现如下问题，查了下，博客园说是因为内库会和conda上的numpy包冲突
import torch报错记录
Python 3.9.12 (main, Jun 1 2022, 06:36:29)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import AutoTokenizer, AutoModel
>>> import torch
OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/
[2] 48685 abort python
于是我在公司电脑(m1)上也简单安装了下再跑transformers，又是正常的。
执行示例
~/Documents/ModelSpace » python shangqing@tianwenyihao
Python 3.9.17 (main, Jul 5 2023, 16:17:03)
[Clang 14.0.6 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
from transformers import AutoTokenizer, AutoModel
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
>>> import torch
>>> sentences = ["样例数据-1", "样例数据-2"]
>>> model = AutoModel.from_pretrained('/Users/shangqing/Documents/ModelSpace/bge-base-en')
>>> encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
>>> with torch.no_grad():
… model_output = model(**encoded_input)
… sentence_embeddings = model_output[0][:, 0]
…
>>> sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
>>> print("Sentence embeddings:", sentence_embeddings)
Sentence embeddings: tensor([[ 0.0097, 0.0307, 0.0027, …, 0.0061, 0.0216, 0.0223],
[ 0.0016, 0.0324, -0.0144, …, -0.0100, 0.0092, 0.0312]])
>>> conda list
File "<stdin>", line 1
conda list
^
SyntaxError: invalid syntax
>>> quit()
/Users/shangqing/opt/anaconda3/envs/hf/lib/python3.9/multiprocessing/resource_tracker.py:96: UserWarning: resource_tracker: process died unexpectedly, relaunching. Some resources might leak.
warnings.warn('resource_tracker: process died unexpectedly, '
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks…
To disable this warning, you can either:
-- Avoid using tokenizers before the fork if possible
-- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Fatal Python error: config_get_locale_encoding: failed to get the locale encoding: nl_langinfo(CODESET) failed
Python runtime state: preinitializedconda list
~/Documents/ModelSpace » conda list shangqing@tianwenyihao
#packages in environment at /Users/shangqing/opt/anaconda3/envs/hf:
#Name Version Build Channel
blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
brotlipy 0.7.0 py39h9ed2024_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates 2023.05.30 hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
certifi 2023.7.22 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cffi 1.15.1 py39h6c40b1e_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
charset-normalizer 2.0.4 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cryptography 41.0.2 py39h3b477ad_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
filelock 3.9.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
fsspec 2023.4.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gmp 6.2.1 he9d5cce_3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
gmpy2 2.1.2 py39hd5de756_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
huggingface_hub 0.15.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
idna 3.4 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
intel-openmp 2023.1.0 ha357a0b_43547 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
jinja2 3.1.2 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libcxx 14.0.6 h9765a3e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi 3.4.4 hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libprotobuf 3.20.3 hfff2838_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuv 1.44.2 h6c40b1e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
markupsafe 2.1.1 py39hca72f7f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl 2023.1.0 h8e150cf_43559 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl-service 2.4.0 py39h6c40b1e_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_fft 1.3.6 py39h07fba90_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mkl_random 1.2.2 py39h07fba90_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mpc 1.1.0 h6ef4df4_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mpfr 4.0.2 h9066e36_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
mpmath 1.3.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ncurses 6.4 hcec6c5f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
networkx 3.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ninja 1.10.2 hecd8cb5_5 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ninja-base 1.10.2 haf03e11_5 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy 1.25.2 py39h827a554_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
numpy-base 1.25.2 py39ha186be2_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
openssl 3.0.10 hca72f7f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
packaging 23.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pip 23.2.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pycparser 2.21 pyhd3eb1b0_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyopenssl 23.2.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pysocks 1.7.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python 3.9.17 h5ee71fb_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pytorch 2.0.1 cpu_py39h77673e4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pyyaml 6.0 py39h6c40b1e_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
readline 8.2 hca72f7f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
regex 2022.7.9 py39hca72f7f_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
requests 2.31.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
setuptools 68.0.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite 3.41.2 h6c40b1e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sympy 1.11.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tbb 2021.8.0 ha357a0b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tk 8.6.12 h5d9f67b_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tokenizers 0.13.2 py39h5240035_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tqdm 4.65.0 py39h01d92e1_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
transformers 4.24.0 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
typing-extensions 4.7.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
typing_extensions 4.7.1 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tzdata 2023c h04d1e81_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
urllib3 1.26.16 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
wheel 0.38.4 py39hecd8cb5_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
xz 5.4.2 h6c40b1e_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
yaml 0.2.5 haf1e3a3_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zlib 1.2.13 h4dc903c_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main这里面出现了几个问题，一是安装git lfs以下载hf模型，brew update响应超时，按掘金的方法使用国内镜像加速，二是下载hf模型后，模型文件还得自己来点击下载才完整，尚不清楚原因。
三是在执行示例时最后的报错，问了chatgpt，也看了下issue，可能是由于tokenizers在并发任务执行时会有问题，暂时应该不用管。
n1t0:
This is happening whenever you use multiprocessing (Often used by data loaders). The way to disable this warning is to set the TOKENIZERS_PARALLELISM environment variable to the value that makes more sense for you. By default, we disable the parallelism to avoid any hidden deadlock that would be hard to debug, but you might be totally fine while keeping it enabled in your specific use-case.
You can try to set it to true, and if your process seems to be stuck, doing nothing, then you should use false.
We'll improve this message to help avoid any confusion (Cf huggingface/tokenizers#328)
我回家后又查了下原因，在stackoverflow上找到了办法并测试成功，出错的原因是因为Conda安装包用Intel MKL数值/科学优化包，MKL有使用OpenMP技术，但macOS有自己的数值/科学计算优化包，并且已经使用了OpenMP，所以会出现OpenMP运行时有多份备份在程序中的报错信息，所以如果要使用Conda的话，得排除MKL支持。
消除OpenMP报错的安装
You should install all packages without MKL support:
conda install nomkl
and then use
conda install numpy scipy pandas tensorflow
followed by
conda remove mkl mkl-service
For more information see conda MKL Optimizations.有了conda的经验后，我直接在requirements.txt后面增加torch和transformers进行安装，命令行使用bge模型没有问题，这么想来一开始可以直接使用pip装transformers，这样就不会有conda的问题。
启动前后端，AI小镇开始跑了，但跑完后总会有一些错误，我内存是16G的，尽可能地用了llama2:13b-text-q8，但还是会有如下的错误：
因未获取有效答案而重试的例子
GNS FUNCTION: <generate_wake_up_hour>
Based on the information provided, Isabella Rodriguez wakes up at 6am.
2023-08-27 16:23:45 GPT_request
prompt:
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on February 14th, 2023, from 5pm to 7pm.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and works at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Isabella's wake up hour:
response:
Based on the information provided, Isabella Rodriguez wakes up at 6am.
finish =======
Based on the information provided, Isabella Rodriguez wakes up at around 6am.
2023-08-27 16:24:10 GPT_request
prompt:
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on February 14th, 2023, from 5pm to 7pm.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and works at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Isabella's wake up hour:
response:
Based on the information provided, Isabella Rodriguez wakes up at 6am.
finish =======
Based on the information provided, Isabella Rodriguez wakes up around 6am.
2023-08-27 16:24:57 GPT_request
prompt:
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on February 14th, 2023, from 5pm to 7pm.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and works at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Isabella's wake up hour:
response:
Based on the information provided, Isabella Rodriguez wakes up around 6am.
finish =======
Based on the information provided, Isabella Rodriguez wakes up at 6am.
2023-08-27 16:25:27 GPT_request
prompt:
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on February 14th, 2023, from 5pm to 7pm.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and works at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Isabella's wake up hour:
response:
Based on the information provided, Isabella Rodriguez wakes up at 6am.
finish =======细细地分析下来，原因是大模型的输出并不是规范的，所以从大模型里获取明确结果需要文本截取的操作，但这操作显然是适配chatGPT，而不是llama2，所以要重新构造一套符合llama2的操作。修改prompt提问方式并修改获取结果的代码后，Isabella就不会每次都是8点起床了
generate_wake_up_hour
GNS FUNCTION:
Based on Isabella Rodriguez's daily plan and lifestyle, she will wake up at [6am].
=== persona/prompt_template/v4_llama2/wake_up_hour_v1.txt
~~~ persona ---------------------------------------------------
Isabella Rodriguez
~~~ gpt_param ----------------------------------------------------
{'engine': 'text-davinci-002', 'max_tokens': 5, 'temperature': 0.8, 'top_p': 1, 'stream': False, 'frequency_penalty': 0, 'presence_penalty': 0, 'stop': ['\n']}
~~~ prompt_input ----------------------------------------------
["Name: Isabella Rodriguez\nAge: 34\nInnate traits: friendly, outgoing, hospitable\nLearned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.\nCurrently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on February 14th, 2023, from 5pm to 7pm.\nLifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.\nDaily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and works at the counter until 8pm, at which point she closes the cafe.\nCurrent Date: Monday February 13\n", 'Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.', 'Isabella']
~~~ prompt ----------------------------------------------------
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on February 14th, 2023, from 5pm to 7pm.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and works at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Whick hour will Isabella wake up? Answer format like [18am].
~~~ output ----------------------------------------------------
6
=== END ==========================================================反复查看代码，可知这里最大的体力活在于提示词模板的列举以及结果的提取上，那复现的时间成本感觉有点高，就暂时先到这里吧。
原文章发布于2023年9月6日