Commit 1f7e0aed authored by jonas's avatar jonas
Browse files

database login

parent 0fd9853b
......@@ -34,7 +34,7 @@ EITHER: download from the PHI Image Database (recommended): https://www2.mps.mpg
Suggested filters for HRT science data:
- **KEYWORD DETECTOR = 'HRT'** <br >
- **Filename\* like \*L1_phi-hrt-ilam\***
- **Filename\* like \*L1_phi-hrt-ilam_date\***
To download via the command line (eg: if you want to save the files on a server and not locally)
```
......@@ -43,7 +43,21 @@ gunzip file.gz
```
Gunzip used to unpack the .gz to the file you want <br>
OR : use download_files.py to download images from the attic repository: https://www2.mps.mpg.de/services/proton/phi/fm/attic/
Can also use `download_from_db.py` to perform multi download from database
Instructions:
1. From the database find the files you wish to download
2. Copy the 'Download File List' that the database will generate
3. Paste into the `file_names.txt` file
4. Create a `.env` file with your MPS windows login: <br>
```text=
USER_NAME =
PHIDATAPASSWORD =
```
5. Set the target download folder in the `download_from_db.py` file
6. Run the file (will require dotenv python module to be installed)
OR : use `download_files.py` to download images from the attic repository: https://www2.mps.mpg.de/services/proton/phi/fm/attic/
## **SETUP**
......
import subprocess
import os
from dotenv import load_dotenv
load_dotenv()
username = os.environ.get('USER_NAME')
password = os.environ.get('PHIDATAPASSWORD')
def get_file_list(text):
text_str = str(text)
text_split = text_str.split("\n")
if text_split[0][0] == 'h':
print("contains https links")
print(text_split)
return text_split
file = open('./file_names.txt')
text = file.read()
text_split = get_file_list(text)
target_directory = '/scratch/slam/sinjan/solo_attic_fits/fits_files/'
os.chdir(target_directory)
print(f"Downloading to target directory: {target_directory}")
for file in text_split:
file = file.split("/")[-1]
if len(file) < 20:
print("Error occured with the file name, skipping to the next one")
pass
else:
subprocess.call(["wget", "--user", f"{username}","--password", f"{password}", f"https://www2.mps.mpg.de/services/proton/phi/imgdb/{file}"])
subprocess.call(["gunzip", f"{file}"])
print("Download and unpacking complete")
solo_L1_phi-hrt-ilam_20200529T205144_V202106111602C_0065260200.fits.gz
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment