README.md 3.49 KB
Newer Older
1
# **SO/PHI-HRT PIPELINE**
jonas's avatar
jonas committed
2

jonas's avatar
jonas committed
3
Reduction software for SO/PHI-HRT instrument on the ESA Solar Orbiter
4
## **PHI-HRT data reduction**
jonas's avatar
jonas committed
5
6
7
8
9
10
11
1. read in science data (+scaling) open path option + open for several scans at once
2. read in flat field (+scaling)- just accepts one flat field fits file
3. read in dark field (+scaling)
4. apply dark field
5. option to clean flat field with unsharp masking (Stokes V only)
6. normalise flat field
7. apply flat field
jonas's avatar
jonas committed
12
8. prefilter correction
jonas's avatar
jonas committed
13
14
9. read in field stop
10. apply field stop
15
16
11. demodulate with const demod matrix <br>
        a) option to output demod to fits file <br>
jonas's avatar
jonas committed
17
12. normalise to quiet sun
18
19
20
21
22
13. calibration <br>
        a) cross talk correction <br>
        (if required) b) ghost correction - **not implemented yet** <br>
14. rte inversion with cmilos <br>
        a) output rte data products to fits file <br>
jonas's avatar
jonas committed
23

jonas's avatar
jonas committed
24

25
#### **CONFIGURATION**
jonas's avatar
jonas committed
26

27
28
29
30
31
32
33
Any and all steps can be turned oon or off as you wish using the keywords in the `phihrt_pipe` function


## **DOWNLOAD INPUT FILES**


EITHER: download from the PHI Image Database (recommended): https://www2.mps.mpg.de/services/proton/phi/imgdb/
34
35

Suggested filters for HRT science data: 
36
- **KEYWORD DETECTOR = 'HRT'** <br >
jonas's avatar
jonas committed
37
- **Filename\* like \*L1_phi-hrt-ilam_date\***
38
        
39
To download via the command line (eg: if you want to save the files on a server and not locally)
40
```
41
wget --user yourusername --password yourpassword file_web_address
42
43
gunzip file.gz
```
44
Gunzip used to unpack the .gz to the file you want  <br>
45

jonas's avatar
jonas committed
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Can also use `download_from_db.py` to perform multi download from database

Instructions:
  1. From the database find the files you wish to download
  2. Copy the 'Download File List' that the database will generate
  3. Paste into the `file_names.txt` file
  4. Create a `.env` file with your MPS windows login: <br> 
      ```text=
      USER_NAME =
      PHIDATAPASSWORD =
      ```  
  5. Set the target download folder in the `download_from_db.py` file
  6. Run the file (will require dotenv python module to be installed) 

OR : use `download_files.py` to download images from the attic repository: https://www2.mps.mpg.de/services/proton/phi/fm/attic/
jonas's avatar
jonas committed
61

62
## **SETUP**
jonas's avatar
jonas committed
63

jonas's avatar
jonas committed
64
1. Compile milos:
jonas's avatar
jonas committed
65

jonas's avatar
jonas committed
66
67
68
69
70
71
72
```bash
make clean
make
```
        
2. Setup virtual environment from requirements.txt

jonas's avatar
jonas committed
73
using pip - REQUIRES PYTHON >= 3.6
jonas's avatar
jonas committed
74
75
76
```bash
pip install -r requirements.txt
```
jonas's avatar
jonas committed
77
using conda (Anaconda3) - creates virtual environment called 'dataproc'
jonas's avatar
jonas committed
78
```bash
jonas's avatar
jonas committed
79
conda env create -f environment.yml
jonas's avatar
jonas committed
80
```
81
2. Change fits files paths, desired processing steps and output directory in ```run.py```
jonas's avatar
jonas committed
82
 
jonas's avatar
jonas committed
83
84
85
86
87
3. Execute ```run.py```

```bash
python run.py
```
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
## **OUTPUT**

#### **Demod File**
Filename: `_reduced.fits `

Shape: [Y,X,POL,WAVE]

#### **RTE products**
- File: `_rte_data_products.fits`

  Shape: [6,Y,X] <br>
  First Index:
  - 0: Continuum Intensity
  - 1: Magnetic Field Strength |B| (Gauss)
  - 2: Inclination (degrees)
  - 3: Azimuth (degrees)
  - 4: Vlos (km/s)
  - 5: Blos (Gauss) </p>


- File: `_blos_rte.fits`

  Shape: [1,Y,X] <br>
  First Index: <br>
  - 0: Blos (Gauss) </p>

- File: `_vlos_rte.fits`

  Shape: [1,Y,X] <br>
  First Index: <br>
  - 0: Vlos (km/s) </p>

- File: `_Icont_rte.fits`

  Shape: [1,Y,X] <br>
  First Index:
  - 0: Continuum Intensity


***

jonas's avatar
jonas committed
129

130
### **Authors**: <br>
jonas's avatar
jonas committed
131

jonas's avatar
jonas committed
132
Jonas Sinjan - Max Planck Institute for Solar System Research, Goettingen, Germany
jonas's avatar
jonas committed
133

134
### **Credit**: <br>
jonas's avatar
jonas committed
135

jonas's avatar
jonas committed
136
137
- SPGPylibs for the foundation, from which it was expanded upon
- CMILOS: RTE INVERSION C code for SOPHI (based on the ILD code MILOS by D. Orozco) Author: juanp (IAA-CSIC)