Atmos Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== ICCP 2024 Cloud Probe Workshop =====

[[.:research:ICCP-CloudProbe2024_Instructions|Instructions for ICCP Cloud Probe Processing Workshop - 2024]]

[[.:research:ICCP-CloudProbe2024_Readme|Readme information about probe types to use]]

===== Probe Processing Notes =====
  * Data directory is /home/Radar_Data/T28_Data/ICCP_Cloud_Workshop (Aircraft/Calgary/Littlestorm)
      * Downloaded synthetic data is stored in ./Jeju_Cloud_Probe_Workshop_Simulations
      * Processed synthetic data using SODA is stored in,
          * ./SODA_gui for files processed using the SODA gui
          * ./ADPAA_commandline for files processed using ADPAA's command line wrapper for SODA

  * Data is processed using the System for OAP Data Analysis (SODA) created by Aaron Bansemer [[https://github.com/abansemer/soda2]]
  * A comparison of the SODA_gui and the ADPAA_commandline was performed to verify that the command line wrapper works properly.
      * The script ./SODA_ADPAA_File_Comparison.py compares the generated file between the ADPAA wrapper and the SODA gui.
          * Example command, ./SODA_ADPAA_File_Comparison.py ADPAA_commandline/HVPS3/20100101_000000_HVPS3_HVPS_ice.pbp.nc SODA_gui/HVPS3/ice_v01_01012010_000000_HVPS3.pbp.nc
      * All variables in the particle-by-particle (pbp) netcdf files are a 100% match except for the rejectionflag. The reason the rejectionflag does not match 100% is because the bin sizes specified in the wrapper are slightly different than the bin sizes specified in the SODA gui (lazy processing on my part). This leads to an out of size range rejection for a lot of smaller particles.

Below, in the right column, are the probenames used in the command line call for SODA. Following the table are the specified methods used in the command line call.
| Probe        | ADPAA-SODA probe_type (Calling Initialized Structure) |
| SPEC 2D-S    | 2ds                                                   |
| SPEC HVPS-3  | hvps                                                  |
| DMT PIP      | pip                                                   |
| DMT CAPS-CIP | cipgs15                                               |

  * The two methods,
    * method="fastcircle"
        * Draws a circle around the particle and uses the circle to size it.
    * method="waterprocessing" 
        * Uses fastcircle, but applies a "stricter roundness criteria to limit processing to particles that may be liquid water. Also apply the Korolev (JTECH, 2007) size correction." -from the SODA manual [[https://github.com/abansemer/soda2/blob/master/MANUAL.pdf]]

To process the synthetic data using the ADPAA wrapper for SODA, navigate into Probe_Processing_Init/ and run the program "run_soda.py". This script is only set up to work with the present directory structure. Comments and some slightly more in depth details on using the wrapper are included as comments in the code. An example command to use the wrapper is ~/CoPAS/ADPAA/src/linkage_soda/initialize_structure/initialize_structure --verbose method='fastcircle' processingtype='allin' probetype='hvps' ../../Jeju_Cloud_Probe_Workshop_Simulations/base100101000000.HVPS_columns_v01 000000 000100 20100101 100

===== Processing Instructions =====
All steps currently operational can be run by executing "File_Runner.py". Some instructions on setting up "File_Runner.py" are inside the code as comments.

  - Extract information from the header. Generate a PbP file that contains the image time from header for each particle, count in buffer, W, L, A, D, P, end diode shadow flag.
    - Program: Extract_Header_Information.py
        - To run on individual file: python3 Extract_Header_Information.py filename
        - Generated file looks like: Output_Files/Step1/20100101_HVPS3_columns_notwater_step1.pbp.nc
    - View/plot with, ncview filename
    - ncplot and ncdump should also work.
  - Determine counts recorded by probe as function of time before any corrections for particle size are made or before the sample volume is computed.
    - Program: Counts_per_second.py
        - To run on individual file: python3 Counts_per_second.py filename
        - Generated file looks like: Output_Files/Step2/20100101_HVPS3_columns_notwater_step2.pbp.nc
    - View/plot with, ncview filename
    - ncplot and ncdump should also work.
  - Determine counts recorded by probe as a function of time after corrections for particle size based on out of focus particles are made (Assume we need to use “water-processing flag in SODA)
    - Program: Counts_per_second.py
        - To run on individual file: python3 Counts_per_second.py filename
        - Generated file looks like: Output_Files/Step3/20100101_HVPS3_columns_notwater_step3.pbp.nc
    - View/plot with, ncview filename
    - ncplot and ncdump should also work.
  - Number distribution functions before corrections for particle size (we will use counts per bin per second so that assumptions about the sample volume dependence on maximum dimension don't affect the results).
    - Program: Number_Distribution.py
        - To run on individual file: python3 Number_Distribution.py filename
        - Generated file looks like: Output_Files/Step4/20100101_HVPS3_columns_notwater_step4.pbp.nc
    - View/plot with, ncview filename
    - ncplot and ncdump should also work.
    - Work is in progress to make matplotlib plots of the distributions for each second.