database login

README.md

@@ -34,7 +34,7 @@ EITHER: download from the PHI Image Database (recommended): https://www2.mps.mpg
 
 Suggested filters for HRT science data: 
 - **KEYWORD DETECTOR = 'HRT'** <br >
-- **Filename\* like \*L1_phi-hrt-ilam\***
+- **Filename\* like \*L1_phi-hrt-ilam_date\***
         
 To download via the command line (eg: if you want to save the files on a server and not locally)
 ```
@@ -43,7 +43,21 @@ gunzip file.gz
 ```
 Gunzip used to unpack the .gz to the file you want  <br>
 
-OR : use download_files.py to download images from the attic repository: https://www2.mps.mpg.de/services/proton/phi/fm/attic/
+Can also use `download_from_db.py` to perform multi download from database
+
+Instructions:
+  1. From the database find the files you wish to download
+  2. Copy the 'Download File List' that the database will generate
+  3. Paste into the `file_names.txt` file
+  4. Create a `.env` file with your MPS windows login: <br> 
+      ```text=
+      USER_NAME =
+      PHIDATAPASSWORD =
+      ```  
+  5. Set the target download folder in the `download_from_db.py` file
+  6. Run the file (will require dotenv python module to be installed) 
+
+OR : use `download_files.py` to download images from the attic repository: https://www2.mps.mpg.de/services/proton/phi/fm/attic/
 
 ## **SETUP**
 

download_from_db.py

+import subprocess
+import os
+from dotenv import load_dotenv
+
+load_dotenv()
+username = os.environ.get('USER_NAME')
+password = os.environ.get('PHIDATAPASSWORD')
+
+def get_file_list(text):
+  text_str = str(text)
+  text_split = text_str.split("\n")
+  if text_split[0][0] == 'h':
+    print("contains https links")
+  print(text_split)
+  return text_split
+
+file = open('./file_names.txt')
+text = file.read()
+
+text_split = get_file_list(text)
+
+target_directory = '/scratch/slam/sinjan/solo_attic_fits/fits_files/'
+
+os.chdir(target_directory)
+
+print(f"Downloading to target directory: {target_directory}")
+
+for file in text_split:
+  file = file.split("/")[-1]
+  if len(file) < 20:
+    print("Error occured with the file name, skipping to the next one")
+    pass
+  else:
+    subprocess.call(["wget", "--user", f"{username}","--password", f"{password}", f"https://www2.mps.mpg.de/services/proton/phi/imgdb/{file}"])
+    subprocess.call(["gunzip", f"{file}"])
+
+print("Download and unpacking complete")

file_names.txt

+solo_L1_phi-hrt-ilam_20200529T205144_V202106111602C_0065260200.fits.gz