Skip to content
Snippets Groups Projects
Commit db7cf401 authored by Christian Boulanger's avatar Christian Boulanger
Browse files

remove installation log

parent 95b65ae7
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
cuda/gemma-finetuning.ipynb
+ 3
− 596
View file @ db7cf401
%% Cell type:markdown id:31c318c0-46dc-44b1-be82-4703028ca744 tags:
# Finetuning Gemma with transformers/torch/cuda/peft
see https://huggingface.co/blog/gemma-peft
%% Cell type:markdown id:6bea4986-2a5b-4c02-bfda-fda113a513b9 tags:
## Install dependencies
%% Cell type:code id:13bfb820-9615-48bd-96a9-5f454f1e67a9 tags:
``` python
!pip install transformers datasets peft python-dotenv accelerate trl
!pip install -i https://pypi.org/simple/ bitsandbytes
!pip install --upgrade bottleneck
#!conda install -c pytorch pytorch -y
```
%% Output
Collecting transformers
Downloading transformers-4.38.2-py3-none-any.whl.metadata (130 kB)
---------------------------------------- 0.0/130.7 kB ? eta -:--:--
--- ------------------------------------ 10.2/130.7 kB ? eta -:--:--
--------------------------- ----------- 92.2/130.7 kB 1.3 MB/s eta 0:00:01
-------------------------------------- 130.7/130.7 kB 1.3 MB/s eta 0:00:00
Collecting datasets
Downloading datasets-2.18.0-py3-none-any.whl.metadata (20 kB)
Collecting peft
Downloading peft-0.9.0-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: python-dotenv in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (0.21.0)
Collecting accelerate
Downloading accelerate-0.27.2-py3-none-any.whl.metadata (18 kB)
Collecting trl
Downloading trl-0.7.11-py3-none-any.whl.metadata (10 kB)
Requirement already satisfied: filelock in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (0.20.2)
Requirement already satisfied: numpy>=1.17 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (1.26.3)
Requirement already satisfied: packaging>=20.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (23.2)
Requirement already satisfied: pyyaml>=5.1 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (2023.12.25)
Requirement already satisfied: requests in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (2.31.0)
Collecting tokenizers<0.19,>=0.14 (from transformers)
Downloading tokenizers-0.15.2-cp311-none-win_amd64.whl.metadata (6.8 kB)
Collecting safetensors>=0.4.1 (from transformers)
Downloading safetensors-0.4.2-cp311-none-win_amd64.whl.metadata (3.9 kB)
Requirement already satisfied: tqdm>=4.27 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from transformers) (4.65.0)
Collecting pyarrow>=12.0.0 (from datasets)
Downloading pyarrow-15.0.0-cp311-cp311-win_amd64.whl.metadata (3.1 kB)
Collecting pyarrow-hotfix (from datasets)
Downloading pyarrow_hotfix-0.6-py3-none-any.whl.metadata (3.6 kB)
Collecting dill<0.3.9,>=0.3.0 (from datasets)
Downloading dill-0.3.8-py3-none-any.whl.metadata (10 kB)
Requirement already satisfied: pandas in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from datasets) (2.1.4)
Collecting xxhash (from datasets)
Downloading xxhash-3.4.1-cp311-cp311-win_amd64.whl.metadata (12 kB)
Collecting multiprocess (from datasets)
Downloading multiprocess-0.70.16-py311-none-any.whl.metadata (7.2 kB)
Requirement already satisfied: fsspec<=2024.2.0,>=2023.1.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from fsspec[http]<=2024.2.0,>=2023.1.0->datasets) (2023.12.2)
Requirement already satisfied: aiohttp in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from datasets) (3.9.1)
Requirement already satisfied: psutil in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from peft) (5.9.0)
Collecting torch>=1.13.0 (from peft)
Downloading torch-2.2.1-cp311-cp311-win_amd64.whl.metadata (26 kB)
Collecting tyro>=0.5.11 (from trl)
Downloading tyro-0.7.3-py3-none-any.whl.metadata (7.7 kB)
Requirement already satisfied: attrs>=17.3.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from aiohttp->datasets) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from aiohttp->datasets) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from aiohttp->datasets) (1.9.4)
Requirement already satisfied: frozenlist>=1.1.1 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from aiohttp->datasets) (1.4.1)
Requirement already satisfied: aiosignal>=1.1.2 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from aiohttp->datasets) (1.3.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from huggingface-hub<1.0,>=0.19.3->transformers) (4.9.0)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from requests->transformers) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from requests->transformers) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from requests->transformers) (1.26.18)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from requests->transformers) (2023.11.17)
Collecting sympy (from torch>=1.13.0->peft)
Downloading sympy-1.12-py3-none-any.whl.metadata (12 kB)
Requirement already satisfied: networkx in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from torch>=1.13.0->peft) (3.2.1)
Requirement already satisfied: jinja2 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from torch>=1.13.0->peft) (3.1.2)
Requirement already satisfied: colorama in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from tqdm>=4.27->transformers) (0.4.6)
Collecting docstring-parser>=0.14.1 (from tyro>=0.5.11->trl)
Downloading docstring_parser-0.15-py3-none-any.whl.metadata (2.4 kB)
Requirement already satisfied: rich>=11.1.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from tyro>=0.5.11->trl) (13.7.0)
Collecting shtab>=1.5.6 (from tyro>=0.5.11->trl)
Downloading shtab-1.7.0-py3-none-any.whl.metadata (7.3 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from pandas->datasets) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from pandas->datasets) (2023.3.post1)
Requirement already satisfied: tzdata>=2022.1 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from pandas->datasets) (2023.3)
Requirement already satisfied: six>=1.5 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (2.15.1)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from jinja2->torch>=1.13.0->peft) (2.1.3)
Collecting mpmath>=0.19 (from sympy->torch>=1.13.0->peft)
Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Requirement already satisfied: mdurl~=0.1 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from markdown-it-py>=2.2.0->rich>=11.1.0->tyro>=0.5.11->trl) (0.1.2)
Downloading transformers-4.38.2-py3-none-any.whl (8.5 MB)
---------------------------------------- 0.0/8.5 MB ? eta -:--:--
-- ------------------------------------- 0.6/8.5 MB 11.8 MB/s eta 0:00:01
------ --------------------------------- 1.5/8.5 MB 18.8 MB/s eta 0:00:01
------------ --------------------------- 2.6/8.5 MB 20.6 MB/s eta 0:00:01
----------------- ---------------------- 3.7/8.5 MB 23.4 MB/s eta 0:00:01
---------------------- ----------------- 4.8/8.5 MB 23.4 MB/s eta 0:00:01
--------------------------- ------------ 5.8/8.5 MB 23.3 MB/s eta 0:00:01
-------------------------------- ------- 7.0/8.5 MB 24.8 MB/s eta 0:00:01
------------------------------------- -- 8.0/8.5 MB 24.5 MB/s eta 0:00:01
--------------------------------------- 8.5/8.5 MB 24.9 MB/s eta 0:00:01
---------------------------------------- 8.5/8.5 MB 21.0 MB/s eta 0:00:00
Downloading datasets-2.18.0-py3-none-any.whl (510 kB)
---------------------------------------- 0.0/510.5 kB ? eta -:--:--
-------------------------------- ------ 419.8/510.5 kB 12.8 MB/s eta 0:00:01
-------------------------------------- - 491.5/510.5 kB 5.1 MB/s eta 0:00:01
--------------------------------------- 501.8/510.5 kB 4.5 MB/s eta 0:00:01
--------------------------------------- 501.8/510.5 kB 4.5 MB/s eta 0:00:01
---------------------------------------- 510.5/510.5 kB 2.7 MB/s eta 0:00:00
Downloading peft-0.9.0-py3-none-any.whl (190 kB)
---------------------------------------- 0.0/190.9 kB ? eta -:--:--
-------------------------------------- - 184.3/190.9 kB ? eta -:--:--
---------------------------------------- 190.9/190.9 kB 5.6 MB/s eta 0:00:00
Downloading accelerate-0.27.2-py3-none-any.whl (279 kB)
---------------------------------------- 0.0/280.0 kB ? eta -:--:--
---------------------------------------- 280.0/280.0 kB 8.7 MB/s eta 0:00:00
Downloading trl-0.7.11-py3-none-any.whl (155 kB)
---------------------------------------- 0.0/155.3 kB ? eta -:--:--
---------------------------------------- 155.3/155.3 kB 4.7 MB/s eta 0:00:00
Downloading dill-0.3.8-py3-none-any.whl (116 kB)
---------------------------------------- 0.0/116.3 kB ? eta -:--:--
---------------------------------------- 116.3/116.3 kB 2.3 MB/s eta 0:00:00
Downloading pyarrow-15.0.0-cp311-cp311-win_amd64.whl (24.8 MB)
---------------------------------------- 0.0/24.8 MB ? eta -:--:--
-- ------------------------------------- 1.5/24.8 MB 31.6 MB/s eta 0:00:01
---- ----------------------------------- 2.5/24.8 MB 27.0 MB/s eta 0:00:01
----- ---------------------------------- 3.6/24.8 MB 28.8 MB/s eta 0:00:01
------- -------------------------------- 4.7/24.8 MB 27.2 MB/s eta 0:00:01
--------- ------------------------------ 5.8/24.8 MB 28.4 MB/s eta 0:00:01
---------- ----------------------------- 6.8/24.8 MB 27.1 MB/s eta 0:00:01
------------ --------------------------- 7.9/24.8 MB 26.6 MB/s eta 0:00:01
-------------- ------------------------- 9.0/24.8 MB 26.2 MB/s eta 0:00:01
---------------- ----------------------- 10.0/24.8 MB 26.7 MB/s eta 0:00:01
----------------- ---------------------- 11.1/24.8 MB 27.3 MB/s eta 0:00:01
------------------- -------------------- 12.1/24.8 MB 26.2 MB/s eta 0:00:01
--------------------- ------------------ 13.1/24.8 MB 26.2 MB/s eta 0:00:01
---------------------- ----------------- 14.1/24.8 MB 27.3 MB/s eta 0:00:01
------------------------ --------------- 15.3/24.8 MB 26.2 MB/s eta 0:00:01
-------------------------- ------------- 16.3/24.8 MB 26.2 MB/s eta 0:00:01
--------------------------- ------------ 17.3/24.8 MB 26.2 MB/s eta 0:00:01
----------------------------- ---------- 18.4/24.8 MB 26.2 MB/s eta 0:00:01
------------------------------- -------- 19.4/24.8 MB 26.2 MB/s eta 0:00:01
-------------------------------- ------- 20.4/24.8 MB 26.2 MB/s eta 0:00:01
---------------------------------- ----- 21.4/24.8 MB 26.2 MB/s eta 0:00:01
------------------------------------ --- 22.4/24.8 MB 25.2 MB/s eta 0:00:01
------------------------------------- -- 23.3/24.8 MB 25.2 MB/s eta 0:00:01
--------------------------------------- 24.3/24.8 MB 25.2 MB/s eta 0:00:01
--------------------------------------- 24.8/24.8 MB 25.2 MB/s eta 0:00:01
--------------------------------------- 24.8/24.8 MB 25.2 MB/s eta 0:00:01
---------------------------------------- 24.8/24.8 MB 20.4 MB/s eta 0:00:00
Downloading safetensors-0.4.2-cp311-none-win_amd64.whl (269 kB)
---------------------------------------- 0.0/269.6 kB ? eta -:--:--
---------------------------------------- 269.6/269.6 kB 8.4 MB/s eta 0:00:00
Downloading tokenizers-0.15.2-cp311-none-win_amd64.whl (2.2 MB)
---------------------------------------- 0.0/2.2 MB ? eta -:--:--
----------------------- ---------------- 1.3/2.2 MB 40.6 MB/s eta 0:00:01
--------------------------------------- 2.2/2.2 MB 28.0 MB/s eta 0:00:01
---------------------------------------- 2.2/2.2 MB 20.0 MB/s eta 0:00:00
Downloading torch-2.2.1-cp311-cp311-win_amd64.whl (198.6 MB)
---------------------------------------- 0.0/198.6 MB ? eta -:--:--
---------------------------------------- 1.1/198.6 MB 35.3 MB/s eta 0:00:06
---------------------------------------- 2.1/198.6 MB 33.0 MB/s eta 0:00:06
--------------------------------------- 3.1/198.6 MB 28.0 MB/s eta 0:00:07
--------------------------------------- 4.0/198.6 MB 28.5 MB/s eta 0:00:07
- -------------------------------------- 5.0/198.6 MB 26.8 MB/s eta 0:00:08
- -------------------------------------- 6.0/198.6 MB 25.4 MB/s eta 0:00:08
- -------------------------------------- 7.0/198.6 MB 24.8 MB/s eta 0:00:08
- -------------------------------------- 7.9/198.6 MB 25.3 MB/s eta 0:00:08
- -------------------------------------- 9.0/198.6 MB 26.1 MB/s eta 0:00:08
-- ------------------------------------- 10.1/198.6 MB 25.9 MB/s eta 0:00:08
-- ------------------------------------- 11.1/198.6 MB 25.2 MB/s eta 0:00:08
-- ------------------------------------- 12.0/198.6 MB 24.2 MB/s eta 0:00:08
-- ------------------------------------- 13.0/198.6 MB 25.2 MB/s eta 0:00:08
-- ------------------------------------- 14.1/198.6 MB 24.2 MB/s eta 0:00:08
--- ------------------------------------ 15.2/198.6 MB 25.1 MB/s eta 0:00:08
--- ------------------------------------ 16.1/198.6 MB 25.2 MB/s eta 0:00:08
--- ------------------------------------ 17.1/198.6 MB 25.1 MB/s eta 0:00:08
--- ------------------------------------ 18.2/198.6 MB 25.1 MB/s eta 0:00:08
--- ------------------------------------ 19.1/198.6 MB 25.2 MB/s eta 0:00:08
---- ----------------------------------- 20.2/198.6 MB 25.1 MB/s eta 0:00:08
---- ----------------------------------- 21.1/198.6 MB 24.2 MB/s eta 0:00:08
---- ----------------------------------- 22.3/198.6 MB 26.2 MB/s eta 0:00:07
---- ----------------------------------- 23.4/198.6 MB 25.1 MB/s eta 0:00:07
---- ----------------------------------- 24.5/198.6 MB 25.1 MB/s eta 0:00:07
----- ---------------------------------- 25.4/198.6 MB 25.2 MB/s eta 0:00:07
----- ---------------------------------- 26.3/198.6 MB 25.1 MB/s eta 0:00:07
----- ---------------------------------- 27.4/198.6 MB 26.2 MB/s eta 0:00:07
----- ---------------------------------- 28.4/198.6 MB 25.2 MB/s eta 0:00:07
----- ---------------------------------- 29.5/198.6 MB 26.2 MB/s eta 0:00:07
------ --------------------------------- 30.6/198.6 MB 26.2 MB/s eta 0:00:07
------ --------------------------------- 31.7/198.6 MB 26.2 MB/s eta 0:00:07
------ --------------------------------- 32.8/198.6 MB 26.2 MB/s eta 0:00:07
------ --------------------------------- 33.9/198.6 MB 25.2 MB/s eta 0:00:07
------- -------------------------------- 34.9/198.6 MB 27.3 MB/s eta 0:00:06
------- -------------------------------- 35.9/198.6 MB 25.1 MB/s eta 0:00:07
------- -------------------------------- 37.0/198.6 MB 26.2 MB/s eta 0:00:07
------- -------------------------------- 38.0/198.6 MB 26.2 MB/s eta 0:00:07
------- -------------------------------- 39.1/198.6 MB 26.2 MB/s eta 0:00:07
-------- ------------------------------- 40.2/198.6 MB 27.3 MB/s eta 0:00:06
-------- ------------------------------- 41.3/198.6 MB 26.2 MB/s eta 0:00:06
-------- ------------------------------- 42.4/198.6 MB 26.2 MB/s eta 0:00:06
-------- ------------------------------- 43.5/198.6 MB 27.3 MB/s eta 0:00:06
-------- ------------------------------- 44.6/198.6 MB 26.2 MB/s eta 0:00:06
--------- ------------------------------ 45.7/198.6 MB 26.2 MB/s eta 0:00:06
--------- ------------------------------ 46.8/198.6 MB 27.3 MB/s eta 0:00:06
--------- ------------------------------ 47.9/198.6 MB 27.3 MB/s eta 0:00:06
--------- ------------------------------ 49.0/198.6 MB 27.3 MB/s eta 0:00:06
---------- ----------------------------- 50.0/198.6 MB 27.3 MB/s eta 0:00:06
---------- ----------------------------- 51.1/198.6 MB 26.2 MB/s eta 0:00:06
---------- ----------------------------- 52.1/198.6 MB 26.2 MB/s eta 0:00:06
---------- ----------------------------- 53.2/198.6 MB 27.3 MB/s eta 0:00:06
---------- ----------------------------- 54.3/198.6 MB 27.3 MB/s eta 0:00:06
----------- ---------------------------- 55.3/198.6 MB 26.2 MB/s eta 0:00:06
----------- ---------------------------- 56.5/198.6 MB 27.3 MB/s eta 0:00:06
----------- ---------------------------- 57.5/198.6 MB 26.2 MB/s eta 0:00:06
----------- ---------------------------- 58.6/198.6 MB 26.2 MB/s eta 0:00:06
------------ --------------------------- 59.7/198.6 MB 27.3 MB/s eta 0:00:06
------------ --------------------------- 60.7/198.6 MB 26.2 MB/s eta 0:00:06
------------ --------------------------- 61.8/198.6 MB 27.3 MB/s eta 0:00:06
------------ --------------------------- 62.9/198.6 MB 27.3 MB/s eta 0:00:05
------------ --------------------------- 64.0/198.6 MB 27.3 MB/s eta 0:00:05
------------- -------------------------- 65.1/198.6 MB 27.3 MB/s eta 0:00:05
------------- -------------------------- 66.2/198.6 MB 27.3 MB/s eta 0:00:05
------------- -------------------------- 67.3/198.6 MB 27.3 MB/s eta 0:00:05
------------- -------------------------- 68.4/198.6 MB 27.3 MB/s eta 0:00:05
------------- -------------------------- 69.4/198.6 MB 27.3 MB/s eta 0:00:05
-------------- ------------------------- 70.5/198.6 MB 27.3 MB/s eta 0:00:05
-------------- ------------------------- 71.6/198.6 MB 27.3 MB/s eta 0:00:05
-------------- ------------------------- 72.7/198.6 MB 26.2 MB/s eta 0:00:05
-------------- ------------------------- 73.8/198.6 MB 27.3 MB/s eta 0:00:05
--------------- ------------------------ 74.9/198.6 MB 27.3 MB/s eta 0:00:05
--------------- ------------------------ 76.0/198.6 MB 27.3 MB/s eta 0:00:05
--------------- ------------------------ 77.1/198.6 MB 26.2 MB/s eta 0:00:05
--------------- ------------------------ 78.2/198.6 MB 26.2 MB/s eta 0:00:05
--------------- ------------------------ 79.3/198.6 MB 26.2 MB/s eta 0:00:05
---------------- ----------------------- 80.4/198.6 MB 27.3 MB/s eta 0:00:05
---------------- ----------------------- 81.5/198.6 MB 27.3 MB/s eta 0:00:05
---------------- ----------------------- 82.6/198.6 MB 26.2 MB/s eta 0:00:05
---------------- ----------------------- 83.6/198.6 MB 26.2 MB/s eta 0:00:05
----------------- ---------------------- 84.7/198.6 MB 27.3 MB/s eta 0:00:05
----------------- ---------------------- 85.8/198.6 MB 27.3 MB/s eta 0:00:05
----------------- ---------------------- 86.9/198.6 MB 28.5 MB/s eta 0:00:04
----------------- ---------------------- 88.0/198.6 MB 27.3 MB/s eta 0:00:05
----------------- ---------------------- 89.1/198.6 MB 26.2 MB/s eta 0:00:05
------------------ --------------------- 90.1/198.6 MB 27.3 MB/s eta 0:00:04
------------------ --------------------- 91.3/198.6 MB 27.3 MB/s eta 0:00:04
------------------ --------------------- 92.3/198.6 MB 26.2 MB/s eta 0:00:05
------------------ --------------------- 93.4/198.6 MB 27.3 MB/s eta 0:00:04
------------------- -------------------- 94.5/198.6 MB 27.3 MB/s eta 0:00:04
------------------- -------------------- 95.4/198.6 MB 27.3 MB/s eta 0:00:04
------------------- -------------------- 96.4/198.6 MB 27.3 MB/s eta 0:00:04
------------------- -------------------- 97.5/198.6 MB 27.3 MB/s eta 0:00:04
------------------- -------------------- 98.6/198.6 MB 27.3 MB/s eta 0:00:04
-------------------- ------------------- 99.7/198.6 MB 26.2 MB/s eta 0:00:04
------------------- ------------------- 100.8/198.6 MB 27.3 MB/s eta 0:00:04
-------------------- ------------------ 101.9/198.6 MB 27.3 MB/s eta 0:00:04
-------------------- ------------------ 103.0/198.6 MB 27.3 MB/s eta 0:00:04
-------------------- ------------------ 104.0/198.6 MB 27.3 MB/s eta 0:00:04
-------------------- ------------------ 105.1/198.6 MB 27.3 MB/s eta 0:00:04
-------------------- ------------------ 106.1/198.6 MB 27.3 MB/s eta 0:00:04
--------------------- ----------------- 107.3/198.6 MB 27.3 MB/s eta 0:00:04
--------------------- ----------------- 108.4/198.6 MB 27.3 MB/s eta 0:00:04
--------------------- ----------------- 109.5/198.6 MB 27.3 MB/s eta 0:00:04
--------------------- ----------------- 110.6/198.6 MB 26.2 MB/s eta 0:00:04
--------------------- ----------------- 111.7/198.6 MB 27.3 MB/s eta 0:00:04
---------------------- ---------------- 112.7/198.6 MB 27.3 MB/s eta 0:00:04
---------------------- ---------------- 113.8/198.6 MB 27.3 MB/s eta 0:00:04
---------------------- ---------------- 114.9/198.6 MB 26.2 MB/s eta 0:00:04
---------------------- ---------------- 116.0/198.6 MB 26.2 MB/s eta 0:00:04
---------------------- ---------------- 117.1/198.6 MB 27.3 MB/s eta 0:00:03
----------------------- --------------- 118.1/198.6 MB 27.3 MB/s eta 0:00:03
----------------------- --------------- 119.2/198.6 MB 27.3 MB/s eta 0:00:03
----------------------- --------------- 120.3/198.6 MB 26.2 MB/s eta 0:00:03
----------------------- --------------- 121.4/198.6 MB 27.3 MB/s eta 0:00:03
------------------------ -------------- 122.4/198.6 MB 26.2 MB/s eta 0:00:03
------------------------ -------------- 123.2/198.6 MB 27.3 MB/s eta 0:00:03
------------------------ -------------- 124.3/198.6 MB 25.2 MB/s eta 0:00:03
------------------------ -------------- 125.3/198.6 MB 25.2 MB/s eta 0:00:03
------------------------ -------------- 126.3/198.6 MB 26.2 MB/s eta 0:00:03
------------------------- ------------- 127.3/198.6 MB 25.2 MB/s eta 0:00:03
------------------------- ------------- 128.2/198.6 MB 24.2 MB/s eta 0:00:03
------------------------- ------------- 129.4/198.6 MB 25.2 MB/s eta 0:00:03
------------------------- ------------- 130.3/198.6 MB 25.2 MB/s eta 0:00:03
------------------------- ------------- 131.2/198.6 MB 24.3 MB/s eta 0:00:03
------------------------- ------------- 132.2/198.6 MB 24.2 MB/s eta 0:00:03
-------------------------- ------------ 132.8/198.6 MB 23.4 MB/s eta 0:00:03
-------------------------- ------------ 133.7/198.6 MB 22.6 MB/s eta 0:00:03
-------------------------- ------------ 134.6/198.6 MB 23.4 MB/s eta 0:00:03
-------------------------- ------------ 135.6/198.6 MB 22.6 MB/s eta 0:00:03
-------------------------- ------------ 136.6/198.6 MB 23.4 MB/s eta 0:00:03
--------------------------- ----------- 137.7/198.6 MB 22.6 MB/s eta 0:00:03
--------------------------- ----------- 138.7/198.6 MB 23.4 MB/s eta 0:00:03
--------------------------- ----------- 139.8/198.6 MB 23.4 MB/s eta 0:00:03
--------------------------- ----------- 140.9/198.6 MB 23.4 MB/s eta 0:00:03
--------------------------- ----------- 142.0/198.6 MB 24.2 MB/s eta 0:00:03
---------------------------- ---------- 143.0/198.6 MB 24.3 MB/s eta 0:00:03
---------------------------- ---------- 144.3/198.6 MB 25.2 MB/s eta 0:00:03
---------------------------- ---------- 145.4/198.6 MB 27.3 MB/s eta 0:00:02
---------------------------- ---------- 146.4/198.6 MB 26.2 MB/s eta 0:00:02
---------------------------- ---------- 147.5/198.6 MB 26.2 MB/s eta 0:00:02
----------------------------- --------- 148.6/198.6 MB 26.2 MB/s eta 0:00:02
----------------------------- --------- 149.7/198.6 MB 26.2 MB/s eta 0:00:02
----------------------------- --------- 150.8/198.6 MB 27.3 MB/s eta 0:00:02
----------------------------- --------- 151.8/198.6 MB 26.2 MB/s eta 0:00:02
------------------------------ -------- 153.0/198.6 MB 26.2 MB/s eta 0:00:02
------------------------------ -------- 154.0/198.6 MB 26.2 MB/s eta 0:00:02
------------------------------ -------- 155.1/198.6 MB 27.3 MB/s eta 0:00:02
------------------------------ -------- 156.2/198.6 MB 27.3 MB/s eta 0:00:02
------------------------------ -------- 157.2/198.6 MB 26.2 MB/s eta 0:00:02
------------------------------- ------- 158.4/198.6 MB 26.2 MB/s eta 0:00:02
------------------------------- ------- 159.5/198.6 MB 27.3 MB/s eta 0:00:02
------------------------------- ------- 160.8/198.6 MB 27.3 MB/s eta 0:00:02
------------------------------- ------- 161.9/198.6 MB 27.3 MB/s eta 0:00:02
-------------------------------- ------ 163.0/198.6 MB 27.3 MB/s eta 0:00:02
-------------------------------- ------ 164.1/198.6 MB 27.3 MB/s eta 0:00:02
-------------------------------- ------ 165.2/198.6 MB 27.3 MB/s eta 0:00:02
-------------------------------- ------ 166.3/198.6 MB 27.3 MB/s eta 0:00:02
-------------------------------- ------ 167.3/198.6 MB 27.3 MB/s eta 0:00:02
--------------------------------- ----- 168.3/198.6 MB 26.2 MB/s eta 0:00:02
--------------------------------- ----- 169.2/198.6 MB 26.2 MB/s eta 0:00:02
--------------------------------- ----- 170.0/198.6 MB 25.1 MB/s eta 0:00:02
--------------------------------- ----- 170.8/198.6 MB 25.2 MB/s eta 0:00:02
--------------------------------- ----- 171.4/198.6 MB 24.2 MB/s eta 0:00:02
--------------------------------- ----- 172.0/198.6 MB 22.6 MB/s eta 0:00:02
--------------------------------- ----- 172.8/198.6 MB 22.6 MB/s eta 0:00:02
---------------------------------- ---- 173.5/198.6 MB 21.1 MB/s eta 0:00:02
---------------------------------- ---- 174.3/198.6 MB 21.1 MB/s eta 0:00:02
---------------------------------- ---- 175.2/198.6 MB 21.1 MB/s eta 0:00:02
---------------------------------- ---- 176.0/198.6 MB 20.5 MB/s eta 0:00:02
---------------------------------- ---- 176.8/198.6 MB 20.5 MB/s eta 0:00:02
---------------------------------- ---- 177.6/198.6 MB 19.8 MB/s eta 0:00:02
----------------------------------- --- 178.4/198.6 MB 19.3 MB/s eta 0:00:02
----------------------------------- --- 179.2/198.6 MB 19.3 MB/s eta 0:00:02
----------------------------------- --- 180.1/198.6 MB 19.3 MB/s eta 0:00:01
----------------------------------- --- 180.9/198.6 MB 19.3 MB/s eta 0:00:01
----------------------------------- --- 181.7/198.6 MB 19.8 MB/s eta 0:00:01
----------------------------------- --- 182.5/198.6 MB 19.8 MB/s eta 0:00:01
----------------------------------- --- 183.3/198.6 MB 20.5 MB/s eta 0:00:01
------------------------------------ -- 184.1/198.6 MB 19.8 MB/s eta 0:00:01
------------------------------------ -- 184.9/198.6 MB 20.5 MB/s eta 0:00:01
------------------------------------ -- 185.7/198.6 MB 19.9 MB/s eta 0:00:01
------------------------------------ -- 186.5/198.6 MB 19.8 MB/s eta 0:00:01
------------------------------------ -- 187.2/198.6 MB 19.9 MB/s eta 0:00:01
------------------------------------ -- 188.1/198.6 MB 19.3 MB/s eta 0:00:01
------------------------------------- - 188.8/198.6 MB 19.9 MB/s eta 0:00:01
------------------------------------- - 189.6/198.6 MB 19.3 MB/s eta 0:00:01
------------------------------------- - 190.4/198.6 MB 19.8 MB/s eta 0:00:01
------------------------------------- - 191.1/198.6 MB 19.3 MB/s eta 0:00:01
------------------------------------- - 192.0/198.6 MB 19.9 MB/s eta 0:00:01
------------------------------------- - 192.8/198.6 MB 19.8 MB/s eta 0:00:01
-------------------------------------- 193.7/198.6 MB 19.3 MB/s eta 0:00:01
-------------------------------------- 194.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 195.4/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 196.3/198.6 MB 19.9 MB/s eta 0:00:01
-------------------------------------- 197.2/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.1/198.6 MB 19.9 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 198.6/198.6 MB 20.5 MB/s eta 0:00:01
--------------------------------------- 198.6/198.6 MB 12.8 MB/s eta 0:00:00
Downloading tyro-0.7.3-py3-none-any.whl (79 kB)
---------------------------------------- 0.0/79.8 kB ? eta -:--:--
----------------------------------- ---- 71.7/79.8 kB ? eta -:--:--
---------------------------------------- 79.8/79.8 kB 2.2 MB/s eta 0:00:00
Downloading multiprocess-0.70.16-py311-none-any.whl (143 kB)
---------------------------------------- 0.0/143.5 kB ? eta -:--:--
---------------------------------------- 143.5/143.5 kB 4.3 MB/s eta 0:00:00
Downloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)
Downloading xxhash-3.4.1-cp311-cp311-win_amd64.whl (29 kB)
Downloading docstring_parser-0.15-py3-none-any.whl (36 kB)
Downloading shtab-1.7.0-py3-none-any.whl (14 kB)
Downloading sympy-1.12-py3-none-any.whl (5.7 MB)
---------------------------------------- 0.0/5.7 MB ? eta -:--:--
------ --------------------------------- 1.0/5.7 MB 20.9 MB/s eta 0:00:01
------------ --------------------------- 1.8/5.7 MB 23.5 MB/s eta 0:00:01
------------------- -------------------- 2.7/5.7 MB 19.3 MB/s eta 0:00:01
------------------------- -------------- 3.6/5.7 MB 20.8 MB/s eta 0:00:01
------------------------------ --------- 4.4/5.7 MB 20.1 MB/s eta 0:00:01
------------------------------------ --- 5.3/5.7 MB 19.8 MB/s eta 0:00:01
--------------------------------------- 5.7/5.7 MB 20.4 MB/s eta 0:00:01
---------------------------------------- 5.7/5.7 MB 18.4 MB/s eta 0:00:00
Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
---------------------------------------- 0.0/536.2 kB ? eta -:--:--
-------------------------------------- 532.5/536.2 kB 32.6 MB/s eta 0:00:01
---------------------------------------- 536.2/536.2 kB 8.3 MB/s eta 0:00:00
Installing collected packages: mpmath, xxhash, sympy, shtab, safetensors, pyarrow-hotfix, pyarrow, docstring-parser, dill, torch, multiprocess, tyro, tokenizers, accelerate, transformers, datasets, trl, peft
Successfully installed accelerate-0.27.2 datasets-2.18.0 dill-0.3.8 docstring-parser-0.15 mpmath-1.3.0 multiprocess-0.70.16 peft-0.9.0 pyarrow-15.0.0 pyarrow-hotfix-0.6 safetensors-0.4.2 shtab-1.7.0 sympy-1.12 tokenizers-0.15.2 torch-2.2.1 transformers-4.38.2 trl-0.7.11 tyro-0.7.3 xxhash-3.4.1
Looking in indexes: https://pypi.org/simple/
Collecting bitsandbytes
Downloading bitsandbytes-0.42.0-py3-none-any.whl.metadata (9.9 kB)
Collecting scipy (from bitsandbytes)
Downloading scipy-1.12.0-cp311-cp311-win_amd64.whl.metadata (60 kB)
---------------------------------------- 0.0/60.4 kB ? eta -:--:--
------------------------- ------------ 41.0/60.4 kB 991.0 kB/s eta 0:00:01
-------------------------------------- 60.4/60.4 kB 643.6 kB/s eta 0:00:00
Requirement already satisfied: numpy<1.29.0,>=1.22.4 in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from scipy->bitsandbytes) (1.26.3)
Downloading bitsandbytes-0.42.0-py3-none-any.whl (105.0 MB)
---------------------------------------- 0.0/105.0 MB ? eta -:--:--
---------------------------------------- 0.4/105.0 MB 8.1 MB/s eta 0:00:13
---------------------------------------- 1.2/105.0 MB 12.9 MB/s eta 0:00:09
--------------------------------------- 2.3/105.0 MB 16.6 MB/s eta 0:00:07
- -------------------------------------- 3.5/105.0 MB 18.5 MB/s eta 0:00:06
- -------------------------------------- 4.8/105.0 MB 20.3 MB/s eta 0:00:05
-- ------------------------------------- 6.0/105.0 MB 21.2 MB/s eta 0:00:05
-- ------------------------------------- 7.2/105.0 MB 22.0 MB/s eta 0:00:05
--- ------------------------------------ 8.5/105.0 MB 22.7 MB/s eta 0:00:05
--- ------------------------------------ 9.7/105.0 MB 23.0 MB/s eta 0:00:05
---- ----------------------------------- 10.8/105.0 MB 25.1 MB/s eta 0:00:04
---- ----------------------------------- 11.1/105.0 MB 24.3 MB/s eta 0:00:04
---- ----------------------------------- 12.1/105.0 MB 23.4 MB/s eta 0:00:04
---- ----------------------------------- 13.1/105.0 MB 24.2 MB/s eta 0:00:04
----- ---------------------------------- 14.1/105.0 MB 23.4 MB/s eta 0:00:04
----- ---------------------------------- 15.2/105.0 MB 23.4 MB/s eta 0:00:04
------ --------------------------------- 16.6/105.0 MB 23.4 MB/s eta 0:00:04
------ --------------------------------- 17.7/105.0 MB 22.6 MB/s eta 0:00:04
------- -------------------------------- 18.8/105.0 MB 23.4 MB/s eta 0:00:04
------- -------------------------------- 19.9/105.0 MB 23.4 MB/s eta 0:00:04
------- -------------------------------- 21.0/105.0 MB 24.3 MB/s eta 0:00:04
-------- ------------------------------- 22.1/105.0 MB 26.2 MB/s eta 0:00:04
-------- ------------------------------- 23.2/105.0 MB 27.3 MB/s eta 0:00:03
--------- ------------------------------ 24.2/105.0 MB 27.3 MB/s eta 0:00:03
--------- ------------------------------ 25.3/105.0 MB 27.3 MB/s eta 0:00:03
---------- ----------------------------- 26.4/105.0 MB 27.3 MB/s eta 0:00:03
---------- ----------------------------- 27.5/105.0 MB 27.3 MB/s eta 0:00:03
---------- ----------------------------- 28.6/105.0 MB 28.4 MB/s eta 0:00:03
----------- ---------------------------- 29.8/105.0 MB 26.2 MB/s eta 0:00:03
----------- ---------------------------- 30.9/105.0 MB 27.3 MB/s eta 0:00:03
------------ --------------------------- 31.9/105.0 MB 27.3 MB/s eta 0:00:03
------------ --------------------------- 33.4/105.0 MB 27.3 MB/s eta 0:00:03
------------- -------------------------- 34.5/105.0 MB 27.3 MB/s eta 0:00:03
------------- -------------------------- 35.6/105.0 MB 27.3 MB/s eta 0:00:03
------------- -------------------------- 36.7/105.0 MB 28.4 MB/s eta 0:00:03
-------------- ------------------------- 37.8/105.0 MB 27.3 MB/s eta 0:00:03
-------------- ------------------------- 38.9/105.0 MB 27.3 MB/s eta 0:00:03
--------------- ------------------------ 40.0/105.0 MB 28.4 MB/s eta 0:00:03
--------------- ------------------------ 41.1/105.0 MB 27.3 MB/s eta 0:00:03
---------------- ----------------------- 42.3/105.0 MB 28.4 MB/s eta 0:00:03
---------------- ----------------------- 43.1/105.0 MB 26.2 MB/s eta 0:00:03
---------------- ----------------------- 44.3/105.0 MB 26.2 MB/s eta 0:00:03
----------------- ---------------------- 45.2/105.0 MB 26.2 MB/s eta 0:00:03
----------------- ---------------------- 46.4/105.0 MB 27.3 MB/s eta 0:00:03
------------------ --------------------- 47.5/105.0 MB 27.3 MB/s eta 0:00:03
------------------ --------------------- 48.8/105.0 MB 26.2 MB/s eta 0:00:03
------------------ --------------------- 49.8/105.0 MB 26.2 MB/s eta 0:00:03
------------------- -------------------- 50.8/105.0 MB 26.2 MB/s eta 0:00:03
------------------- -------------------- 51.6/105.0 MB 25.1 MB/s eta 0:00:03
-------------------- ------------------- 52.9/105.0 MB 25.2 MB/s eta 0:00:03
-------------------- ------------------- 54.1/105.0 MB 25.2 MB/s eta 0:00:03
--------------------- ------------------ 55.4/105.0 MB 25.2 MB/s eta 0:00:02
--------------------- ------------------ 56.6/105.0 MB 25.2 MB/s eta 0:00:02
---------------------- ----------------- 57.9/105.0 MB 25.1 MB/s eta 0:00:02
---------------------- ----------------- 59.2/105.0 MB 25.1 MB/s eta 0:00:02
----------------------- ---------------- 60.4/105.0 MB 26.2 MB/s eta 0:00:02
----------------------- ---------------- 61.7/105.0 MB 26.2 MB/s eta 0:00:02
----------------------- ---------------- 62.9/105.0 MB 27.3 MB/s eta 0:00:02
------------------------ --------------- 64.2/105.0 MB 27.3 MB/s eta 0:00:02
------------------------ --------------- 65.5/105.0 MB 27.3 MB/s eta 0:00:02
------------------------- -------------- 66.5/105.0 MB 27.3 MB/s eta 0:00:02
------------------------- -------------- 67.7/105.0 MB 28.5 MB/s eta 0:00:02
-------------------------- ------------- 68.8/105.0 MB 28.5 MB/s eta 0:00:02
-------------------------- ------------- 69.9/105.0 MB 27.3 MB/s eta 0:00:02
--------------------------- ------------ 70.9/105.0 MB 28.5 MB/s eta 0:00:02
--------------------------- ------------ 71.9/105.0 MB 27.3 MB/s eta 0:00:02
--------------------------- ------------ 72.9/105.0 MB 26.2 MB/s eta 0:00:02
---------------------------- ----------- 73.7/105.0 MB 26.2 MB/s eta 0:00:02
---------------------------- ----------- 74.0/105.0 MB 26.2 MB/s eta 0:00:02
---------------------------- ----------- 75.1/105.0 MB 24.2 MB/s eta 0:00:02
----------------------------- ---------- 76.2/105.0 MB 24.3 MB/s eta 0:00:02
----------------------------- ---------- 77.3/105.0 MB 23.4 MB/s eta 0:00:02
----------------------------- ---------- 78.3/105.0 MB 24.2 MB/s eta 0:00:02
------------------------------ --------- 78.9/105.0 MB 23.4 MB/s eta 0:00:02
------------------------------ --------- 80.3/105.0 MB 22.6 MB/s eta 0:00:02
------------------------------ --------- 81.2/105.0 MB 23.4 MB/s eta 0:00:02
------------------------------- -------- 81.8/105.0 MB 21.9 MB/s eta 0:00:02
------------------------------- -------- 82.5/105.0 MB 21.1 MB/s eta 0:00:02
------------------------------- -------- 83.3/105.0 MB 21.8 MB/s eta 0:00:01
-------------------------------- ------- 84.1/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------- ------- 84.9/105.0 MB 22.6 MB/s eta 0:00:01
-------------------------------- ------- 85.8/105.0 MB 21.8 MB/s eta 0:00:01
--------------------------------- ------ 86.6/105.0 MB 21.1 MB/s eta 0:00:01
--------------------------------- ------ 87.5/105.0 MB 21.1 MB/s eta 0:00:01
--------------------------------- ------ 88.3/105.0 MB 20.5 MB/s eta 0:00:01
--------------------------------- ------ 89.1/105.0 MB 19.9 MB/s eta 0:00:01
---------------------------------- ----- 89.9/105.0 MB 19.8 MB/s eta 0:00:01
---------------------------------- ----- 90.8/105.0 MB 19.8 MB/s eta 0:00:01
---------------------------------- ----- 91.6/105.0 MB 20.5 MB/s eta 0:00:01
----------------------------------- ---- 92.3/105.0 MB 19.8 MB/s eta 0:00:01
----------------------------------- ---- 93.2/105.0 MB 20.5 MB/s eta 0:00:01
----------------------------------- ---- 94.0/105.0 MB 20.5 MB/s eta 0:00:01
------------------------------------ --- 94.6/105.0 MB 19.8 MB/s eta 0:00:01
------------------------------------ --- 95.5/105.0 MB 20.5 MB/s eta 0:00:01
------------------------------------ --- 96.3/105.0 MB 19.8 MB/s eta 0:00:01
------------------------------------- -- 97.2/105.0 MB 20.5 MB/s eta 0:00:01
------------------------------------- -- 98.0/105.0 MB 19.8 MB/s eta 0:00:01
------------------------------------- -- 98.9/105.0 MB 19.8 MB/s eta 0:00:01
-------------------------------------- - 99.8/105.0 MB 20.5 MB/s eta 0:00:01
------------------------------------- - 100.6/105.0 MB 20.5 MB/s eta 0:00:01
------------------------------------- - 101.6/105.0 MB 19.8 MB/s eta 0:00:01
-------------------------------------- 102.4/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 103.3/105.0 MB 20.5 MB/s eta 0:00:01
-------------------------------------- 104.1/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 105.0/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 105.0/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 105.0/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 105.0/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 105.0/105.0 MB 21.1 MB/s eta 0:00:01
-------------------------------------- 105.0/105.0 MB 21.1 MB/s eta 0:00:01
--------------------------------------- 105.0/105.0 MB 14.5 MB/s eta 0:00:00
Downloading scipy-1.12.0-cp311-cp311-win_amd64.whl (46.2 MB)
---------------------------------------- 0.0/46.2 MB ? eta -:--:--
- -------------------------------------- 1.2/46.2 MB 25.1 MB/s eta 0:00:02
- -------------------------------------- 2.1/46.2 MB 26.4 MB/s eta 0:00:02
-- ------------------------------------- 2.9/46.2 MB 23.3 MB/s eta 0:00:02
--- ------------------------------------ 3.8/46.2 MB 24.4 MB/s eta 0:00:02
---- ----------------------------------- 4.7/46.2 MB 23.3 MB/s eta 0:00:02
---- ----------------------------------- 5.6/46.2 MB 22.4 MB/s eta 0:00:02
----- ---------------------------------- 6.5/46.2 MB 22.0 MB/s eta 0:00:02
------ --------------------------------- 7.4/46.2 MB 22.6 MB/s eta 0:00:02
------- -------------------------------- 8.3/46.2 MB 23.2 MB/s eta 0:00:02
------- -------------------------------- 9.1/46.2 MB 22.4 MB/s eta 0:00:02
-------- ------------------------------- 10.2/46.2 MB 22.4 MB/s eta 0:00:02
--------- ------------------------------ 10.9/46.2 MB 21.8 MB/s eta 0:00:02
---------- ----------------------------- 11.7/46.2 MB 21.8 MB/s eta 0:00:02
---------- ----------------------------- 12.6/46.2 MB 21.1 MB/s eta 0:00:02
----------- ---------------------------- 13.3/46.2 MB 21.8 MB/s eta 0:00:02
------------ --------------------------- 14.2/46.2 MB 21.1 MB/s eta 0:00:02
------------ --------------------------- 14.8/46.2 MB 21.1 MB/s eta 0:00:02
------------- -------------------------- 15.5/46.2 MB 20.5 MB/s eta 0:00:02
------------- -------------------------- 16.1/46.2 MB 19.8 MB/s eta 0:00:02
-------------- ------------------------- 16.8/46.2 MB 19.9 MB/s eta 0:00:02
--------------- ------------------------ 17.5/46.2 MB 19.3 MB/s eta 0:00:02
--------------- ------------------------ 18.1/46.2 MB 18.2 MB/s eta 0:00:02
--------------- ------------------------ 18.5/46.2 MB 18.2 MB/s eta 0:00:02
---------------- ----------------------- 19.5/46.2 MB 18.2 MB/s eta 0:00:02
----------------- ---------------------- 20.2/46.2 MB 18.2 MB/s eta 0:00:02
------------------ --------------------- 20.9/46.2 MB 17.7 MB/s eta 0:00:02
------------------ --------------------- 21.5/46.2 MB 17.2 MB/s eta 0:00:02
------------------- -------------------- 22.1/46.2 MB 16.8 MB/s eta 0:00:02
------------------- -------------------- 22.8/46.2 MB 16.4 MB/s eta 0:00:02
-------------------- ------------------- 23.4/46.2 MB 16.4 MB/s eta 0:00:02
-------------------- ------------------- 24.2/46.2 MB 16.4 MB/s eta 0:00:02
--------------------- ------------------ 24.8/46.2 MB 16.0 MB/s eta 0:00:02
---------------------- ----------------- 25.5/46.2 MB 16.0 MB/s eta 0:00:02
---------------------- ----------------- 26.2/46.2 MB 16.8 MB/s eta 0:00:02
----------------------- ---------------- 26.9/46.2 MB 16.4 MB/s eta 0:00:02
----------------------- ---------------- 27.6/46.2 MB 16.8 MB/s eta 0:00:02
------------------------ --------------- 28.3/46.2 MB 16.8 MB/s eta 0:00:02
------------------------- -------------- 29.1/46.2 MB 16.8 MB/s eta 0:00:02
------------------------- -------------- 29.8/46.2 MB 16.8 MB/s eta 0:00:01
-------------------------- ------------- 30.5/46.2 MB 16.8 MB/s eta 0:00:01
--------------------------- ------------ 31.3/46.2 MB 17.2 MB/s eta 0:00:01
--------------------------- ------------ 32.0/46.2 MB 17.3 MB/s eta 0:00:01
---------------------------- ----------- 32.7/46.2 MB 17.3 MB/s eta 0:00:01
----------------------------- ---------- 33.5/46.2 MB 17.7 MB/s eta 0:00:01
----------------------------- ---------- 34.2/46.2 MB 17.3 MB/s eta 0:00:01
------------------------------ --------- 34.9/46.2 MB 17.7 MB/s eta 0:00:01
------------------------------ --------- 35.7/46.2 MB 17.7 MB/s eta 0:00:01
------------------------------- -------- 36.4/46.2 MB 17.7 MB/s eta 0:00:01
-------------------------------- ------- 37.2/46.2 MB 18.2 MB/s eta 0:00:01
-------------------------------- ------- 38.0/46.2 MB 18.2 MB/s eta 0:00:01
--------------------------------- ------ 38.7/46.2 MB 18.2 MB/s eta 0:00:01
---------------------------------- ----- 39.4/46.2 MB 18.2 MB/s eta 0:00:01
---------------------------------- ----- 40.2/46.2 MB 18.2 MB/s eta 0:00:01
----------------------------------- ---- 41.0/46.2 MB 18.7 MB/s eta 0:00:01
------------------------------------ --- 41.8/46.2 MB 18.2 MB/s eta 0:00:01
------------------------------------ --- 42.6/46.2 MB 18.2 MB/s eta 0:00:01
------------------------------------- -- 43.4/46.2 MB 19.3 MB/s eta 0:00:01
-------------------------------------- - 44.2/46.2 MB 18.7 MB/s eta 0:00:01
-------------------------------------- - 44.9/46.2 MB 19.3 MB/s eta 0:00:01
--------------------------------------- 45.7/46.2 MB 18.7 MB/s eta 0:00:01
--------------------------------------- 46.2/46.2 MB 18.7 MB/s eta 0:00:01
--------------------------------------- 46.2/46.2 MB 18.7 MB/s eta 0:00:01
--------------------------------------- 46.2/46.2 MB 18.7 MB/s eta 0:00:01
---------------------------------------- 46.2/46.2 MB 15.9 MB/s eta 0:00:00
Installing collected packages: scipy, bitsandbytes
Successfully installed bitsandbytes-0.42.0 scipy-1.12.0
Requirement already satisfied: bottleneck in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (1.3.5)
Collecting bottleneck
Downloading Bottleneck-1.3.8-cp311-cp311-win_amd64.whl.metadata (8.1 kB)
Requirement already satisfied: numpy in c:\users\boulanger\appdata\local\miniconda3\lib\site-packages (from bottleneck) (1.26.3)
Downloading Bottleneck-1.3.8-cp311-cp311-win_amd64.whl (110 kB)
---------------------------------------- 0.0/110.1 kB ? eta -:--:--
---------- ---------------------------- 30.7/110.1 kB 660.6 kB/s eta 0:00:01
------------------------------------- -- 102.4/110.1 kB 1.5 MB/s eta 0:00:01
------------------------------------- -- 102.4/110.1 kB 1.5 MB/s eta 0:00:01
-------------------------------------- 110.1/110.1 kB 799.1 kB/s eta 0:00:00
Installing collected packages: bottleneck
Attempting uninstall: bottleneck
Found existing installation: Bottleneck 1.3.5
Uninstalling Bottleneck-1.3.5:
Successfully uninstalled Bottleneck-1.3.5
Successfully installed bottleneck-1.3.8
%% Cell type:markdown id:33bb741c-e55d-4403-bf35-0c303dfce389 tags:
## Show info on hardware
%% Cell type:code id:2588a200-6d74-476a-b0f3-a046bc2bb332 tags:
``` python
import os
import psutil
# RAM
ram_bytes = psutil.virtual_memory().total
ram_gb = ram_bytes / (1024**3)
print(f"Total RAM: {ram_gb:.2f} GB")
# CPU cores
cpu_cores = os.cpu_count()
print(f"Total CPU Cores: {cpu_cores}")
```
%% Cell type:markdown id:09e29396-1c1e-44ec-9e74-54a67873e91a tags:
## Download and quantize model
%% Cell type:code id:6b625baf-d995-43b8-b8ce-f95ec624c73a tags:
``` python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from dotenv import load_dotenv
load_dotenv('env.txt')
model_id = "google/gemma-2b"
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained(model_id, token=os.environ['HF_TOKEN'], padding_side='right')
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0}, token=os.environ['HF_TOKEN'])
```
%% Cell type:code id:dd0a49a1-9aeb-47e4-b19a-9a9bb2e4bcad tags:
``` python
text = "Quote: Imagination is more"
device = "cuda:0"
inputs = tokenizer(text, return_tensors="pt").to(device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
%% Cell type:code id:b31b170e-280a-4742-8ef4-73b45ee87927 tags:
``` python
from datasets import load_dataset
data = load_dataset("Abirate/english_quotes")
data = data.map(lambda samples: tokenizer(samples["quote"]), batched=True)
```
%% Cell type:code id:1839f1e8-b0d8-4d45-859b-011d2bf1f146 tags:
``` python
import transformers
from trl import SFTTrainer
from peft import LoraConfig
lora_config = LoraConfig(
r=8,
target_modules=["q_proj", "o_proj", "k_proj", "v_proj", "gate_proj", "up_proj", "down_proj"],
task_type="CAUSAL_LM",
)
def formatting_func(example):
text = f"Quote: {example['quote'][0]}\nAuthor: {example['author'][0]}"
return [text]
trainer = SFTTrainer(
model=model,
train_dataset=data["train"],
max_seq_length=1024,
args=transformers.TrainingArguments(
per_device_train_batch_size=1,
gradient_accumulation_steps=4,
warmup_steps=2,
max_steps=10,
learning_rate=2e-4,
fp16=True,
logging_steps=1,
output_dir="outputs",
optim="paged_adamw_8bit"
),
peft_config=lora_config,
formatting_func=formatting_func,
)
trainer.train()
```
%% Output
/u/cboul/.local/lib/python3.10/site-packages/trl/trainer/sft_trainer.py:294: UserWarning: You passed a tokenizer with `padding_side` not equal to `right` to the SFTTrainer. This might lead to some unexpected behaviour due to overflow issues when training a model in half-precision. You might consider adding `tokenizer.padding_side = 'right'` to your code.
warnings.warn(
---------------------------------------------------------------------------
OutOfMemoryError Traceback (most recent call last)
Cell In[4], line 33
13 return [text]
15 trainer = SFTTrainer(
16 model=model,
17 train_dataset=data["train"],
(...)
31 formatting_func=formatting_func,
32 )
---> 33 trainer.train()
File ~/.local/lib/python3.10/site-packages/trl/trainer/sft_trainer.py:331, in SFTTrainer.train(self, *args, **kwargs)
328 if self.neftune_noise_alpha is not None and not self._trainer_supports_neftune:
329 self.model = self._trl_activate_neftune(self.model)
--> 331 output = super().train(*args, **kwargs)
333 # After training we make sure to retrieve back the original forward pass method
334 # for the embedding layer by removing the forward post hook.
335 if self.neftune_noise_alpha is not None and not self._trainer_supports_neftune:
File ~/.local/lib/python3.10/site-packages/transformers/trainer.py:1624, in Trainer.train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
1622 hf_hub_utils.enable_progress_bars()
1623 else:
-> 1624 return inner_training_loop(
1625 args=args,
1626 resume_from_checkpoint=resume_from_checkpoint,
1627 trial=trial,
1628 ignore_keys_for_eval=ignore_keys_for_eval,
1629 )
File ~/.local/lib/python3.10/site-packages/transformers/trainer.py:1961, in Trainer._inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
1958 self.control = self.callback_handler.on_step_begin(args, self.state, self.control)
1960 with self.accelerator.accumulate(model):
-> 1961 tr_loss_step = self.training_step(model, inputs)
1963 if (
1964 args.logging_nan_inf_filter
1965 and not is_torch_tpu_available()
1966 and (torch.isnan(tr_loss_step) or torch.isinf(tr_loss_step))
1967 ):
1968 # if loss is nan or inf simply add the average of previous logged losses
1969 tr_loss += tr_loss / (1 + self.state.global_step - self._globalstep_last_logged)
File ~/.local/lib/python3.10/site-packages/transformers/trainer.py:2902, in Trainer.training_step(self, model, inputs)
2899 return loss_mb.reduce_mean().detach().to(self.args.device)
2901 with self.compute_loss_context_manager():
-> 2902 loss = self.compute_loss(model, inputs)
2904 if self.args.n_gpu > 1:
2905 loss = loss.mean() # mean() to average on multi-gpu parallel training
File ~/.local/lib/python3.10/site-packages/transformers/trainer.py:2925, in Trainer.compute_loss(self, model, inputs, return_outputs)
2923 else:
2924 labels = None
-> 2925 outputs = model(**inputs)
2926 # Save past state if it exists
2927 # TODO: this needs to be fixed and made cleaner later.
2928 if self.args.past_index >= 0:
File ~/.local/lib/python3.10/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
1509 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1510 else:
-> 1511 return self._call_impl(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
1515 # If we don't have any hooks, we want to skip the rest of the logic in
1516 # this function, and just call forward.
1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1518 or _global_backward_pre_hooks or _global_backward_hooks
1519 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520 return forward_call(*args, **kwargs)
1522 try:
1523 result = None
File ~/.local/lib/python3.10/site-packages/accelerate/utils/operations.py:817, in convert_outputs_to_fp32.<locals>.forward(*args, **kwargs)
816 def forward(*args, **kwargs):
--> 817 return model_forward(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/accelerate/utils/operations.py:805, in ConvertOutputsToFp32.__call__(self, *args, **kwargs)
804 def __call__(self, *args, **kwargs):
--> 805 return convert_to_fp32(self.model_forward(*args, **kwargs))
File ~/.local/lib/python3.10/site-packages/torch/amp/autocast_mode.py:16, in autocast_decorator.<locals>.decorate_autocast(*args, **kwargs)
13 @functools.wraps(func)
14 def decorate_autocast(*args, **kwargs):
15 with autocast_instance:
---> 16 return func(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/peft/peft_model.py:1091, in PeftModelForCausalLM.forward(self, input_ids, attention_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict, task_ids, **kwargs)
1089 if peft_config.peft_type == PeftType.POLY:
1090 kwargs["task_ids"] = task_ids
-> 1091 return self.base_model(
1092 input_ids=input_ids,
1093 attention_mask=attention_mask,
1094 inputs_embeds=inputs_embeds,
1095 labels=labels,
1096 output_attentions=output_attentions,
1097 output_hidden_states=output_hidden_states,
1098 return_dict=return_dict,
1099 **kwargs,
1100 )
1102 batch_size = _get_batch_size(input_ids, inputs_embeds)
1103 if attention_mask is not None:
1104 # concat prompt attention mask
File ~/.local/lib/python3.10/site-packages/torch/nn/modules/module.py:1511, in Module._wrapped_call_impl(self, *args, **kwargs)
1509 return self._compiled_call_impl(*args, **kwargs) # type: ignore[misc]
1510 else:
-> 1511 return self._call_impl(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/torch/nn/modules/module.py:1520, in Module._call_impl(self, *args, **kwargs)
1515 # If we don't have any hooks, we want to skip the rest of the logic in
1516 # this function, and just call forward.
1517 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1518 or _global_backward_pre_hooks or _global_backward_hooks
1519 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1520 return forward_call(*args, **kwargs)
1522 try:
1523 result = None
File ~/.local/lib/python3.10/site-packages/peft/tuners/tuners_utils.py:160, in BaseTuner.forward(self, *args, **kwargs)
159 def forward(self, *args: Any, **kwargs: Any):
--> 160 return self.model.forward(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/accelerate/hooks.py:166, in add_hook_to_module.<locals>.new_forward(module, *args, **kwargs)
164 output = module._old_forward(*args, **kwargs)
165 else:
--> 166 output = module._old_forward(*args, **kwargs)
167 return module._hf_hook.post_forward(module, output)
File ~/.local/lib/python3.10/site-packages/transformers/models/gemma/modeling_gemma.py:1088, in GemmaForCausalLM.forward(self, input_ids, attention_mask, position_ids, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict, cache_position)
1086 hidden_states = outputs[0]
1087 logits = self.lm_head(hidden_states)
-> 1088 logits = logits.float()
1089 loss = None
1090 if labels is not None:
1091 # Shift so that tokens < n predict n
OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB. GPU 0 has a total capacity of 23.67 GiB of which 38.19 MiB is free. Process 36053 has 11.64 GiB memory in use. Process 31527 has 7.32 GiB memory in use. Including non-PyTorch memory, this process has 4.63 GiB memory in use. Of the allocated memory 4.34 GiB is allocated by PyTorch, and 9.73 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
%% Cell type:code id:4c531e12-f314-47a4-bcde-f6a485c2891e tags:
``` python
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment