Differences

This shows you the differences between two versions of the page.

--- atmos:citation:research:iccp-cloudprobe2024 [2024/07/30 17:56] – klinman
+++ atmos:citation:research:iccp-cloudprobe2024 [2024/07/31 20:50] (current) – klinman
@@ Line 1: / Line 1: @@
 ===== ICCP 2024 Cloud Probe Workshop =====
-[[.:research:ICCP-CloudProbe2024_Instructions|Instrumection for ICCP Cloud Probe Processing Workshop - 2024]]
+[[.:research:ICCP-CloudProbe2024_Instructions|Instructions for ICCP Cloud Probe Processing Workshop - 2024]]
-[[.:research:ICCP-CloudProbe2024_Readme|Readme information about probe types to use]]
-===== Probe Processing Notes =====
-  - Synthetic data is in the directory /home/Radar_Data/T28_Data/ICCP_Cloud_Workshop (Aircraft/Calgary/Littlestorm)
-  - Downloaded the Input/Simulation data
-    * Stored under Jeju_Cloud_Probe_Workshop_Simulations.
-  - Comparison between ADPAA scripted output and SODA GNU generated files.
-    * Run - python3 /home/Radar_Data/T28_Data/ICCP_Cloud_Workshop/SODA_ADPAA_File_Comparison.py
-    * When not 100 % match on rejectionflag is because of bins size difference.
-| Probe        | ADPAA-SODA probe_type (Calling Initialized Structure) |
-| SPEC 2D-S    | 2ds                                                   |
-| Spec HVPS-3  | hvps                                                  |
-| DMT PIP      | pip                                                   |
-| DMT CAPS-CIP | cipgs15                                               |
-  * Do two methods
-    * Use ADPAA method="fastcircle"
-    * Use method="waterprocessing"
-==== PROCESSING INSTRUCTIONS COMMENTS (Number Same as Instructions ====
-  - Extract information from the header. Generate a PbP file that contains the image time from header for each particle, count in buffer, W, L, A, D, P, end diode shadow flag.
-    - HVPS3
-      - ~/CoPAS/ADPAA/src/linkage_soda/initialize_structure/initialize_structure --verbose method='xsize' processingtype='allin' probetype='hvps' ../../Jeju_Cloud_Probe_Workshop_Simulations/base100101000000.HVPS_columns_v01 000000 000100 20100101 100
-    - PIP
-      - ~/CoPAS/ADPAA/src/linkage_soda/initialize_structure/initialize_structure --verbose method='xsize' processingtype='allin' probetype='pip' ../../Jeju_Cloud_Probe_Workshop_Simulations/base100101000000.PIP_columns_v01 000000 000100 20100101 100
-    - Run python3 Extraxt_Header_Information.py
-    - Creates Output_Files/Steps/20100101_HVPS3_columns_step1.pbp.nc
-    - View/Plot with ncview 20100101_HVPS3_columns_step1.pbp.nc
-    - ncplot and ncdump should also work.
-  - Determine counts recorded by probe as function of time before any corrections for particle size are made or before the sample volume is computed.
-    - run python3 Counts_per_second.py
-    - Creates files Steps/20100101_HVPS3_columns_step2.pbp.nc
-    - Need to add plotting to programming and add resulting plots here.
-===== ICCP 2024 Cloud Probe Workshop =====
-[[.:research:ICCP-CloudProbe2024_Instructions|Instructiond for ICCP Cloud Probe Processing Workshop - 2024]]
 [[.:research:ICCP-CloudProbe2024_Readme|Readme information about probe types to use]]
@@ Line 54: / Line 12: @@
           * ./ADPAA_commandline for files processed using ADPAA's command line wrapper for SODA
+  * Data is processed using the System for OAP Data Analysis (SODA) created by Aaron Bansemer [[https://github.com/abansemer/soda2]]
   * A comparison of the SODA_gui and the ADPAA_commandline was performed to verify that the command line wrapper works properly.
       * The script ./SODA_ADPAA_File_Comparison.py compares the generated file between the ADPAA wrapper and the SODA gui.
+          * Example command, ./SODA_ADPAA_File_Comparison.py ADPAA_commandline/HVPS3/20100101_000000_HVPS3_HVPS_ice.pbp.nc SODA_gui/HVPS3/ice_v01_01012010_000000_HVPS3.pbp.nc
       * All variables in the particle-by-particle (pbp) netcdf files are a 100% match except for the rejectionflag. The reason the rejectionflag does not match 100% is because the bin sizes specified in the wrapper are slightly different than the bin sizes specified in the SODA gui (lazy processing on my part). This leads to an out of size range rejection for a lot of smaller particles.
@@ Line 61: / Line 21: @@
 | Probe        | ADPAA-SODA probe_type (Calling Initialized Structure) |
 | SPEC 2D-S    | 2ds                                                   |
-| Spec HVPS-3  | hvps                                                  |
+| SPEC HVPS-3  | hvps                                                  |
 | DMT PIP      | pip                                                   |
 | DMT CAPS-CIP | cipgs15                                               |
@@ Line 70: / Line 30: @@
     * method="waterprocessing"
         * Uses fastcircle, but applies a "stricter roundness criteria to limit processing to particles that may be liquid water. Also apply the Korolev (JTECH, 2007) size correction." -from the SODA manual [[https://github.com/abansemer/soda2/blob/master/MANUAL.pdf]]
+To process the synthetic data using the ADPAA wrapper for SODA, navigate into Probe_Processing_Init/ and run the program "run_soda.py". This script is only set up to work with the present directory structure. Comments and some slightly more in depth details on using the wrapper are included as comments in the code. An example command to use the wrapper is ~/CoPAS/ADPAA/src/linkage_soda/initialize_structure/initialize_structure --verbose method='fastcircle' processingtype='allin' probetype='hvps' ../../Jeju_Cloud_Probe_Workshop_Simulations/base100101000000.HVPS_columns_v01 000000 000100 20100101 100
+===== Processing Instructions =====
+All steps currently operational can be run by executing "File_Runner.py". Some instructions on setting up "File_Runner.py" are inside the code as comments.
+  - Extract information from the header. Generate a PbP file that contains the image time from header for each particle, count in buffer, W, L, A, D, P, end diode shadow flag.
+    - Program: Extract_Header_Information.py
+        - To run on individual file: python3 Extract_Header_Information.py filename
+        - Generated file looks like: Output_Files/Step1/20100101_HVPS3_columns_notwater_step1.pbp.nc
+    - View/plot with, ncview filename
+    - ncplot and ncdump should also work.
+  - Determine counts recorded by probe as function of time before any corrections for particle size are made or before the sample volume is computed.
+    - Program: Counts_per_second.py
+        - To run on individual file: python3 Counts_per_second.py filename
+        - Generated file looks like: Output_Files/Step2/20100101_HVPS3_columns_notwater_step2.pbp.nc
+    - View/plot with, ncview filename
+    - ncplot and ncdump should also work.
+  - Determine counts recorded by probe as a function of time after corrections for particle size based on out of focus particles are made (Assume we need to use “water-processing flag in SODA)
+    - Program: Counts_per_second.py
+        - To run on individual file: python3 Counts_per_second.py filename
+        - Generated file looks like: Output_Files/Step3/20100101_HVPS3_columns_notwater_step3.pbp.nc
+    - View/plot with, ncview filename
+    - ncplot and ncdump should also work.
+  - Number distribution functions before corrections for particle size (we will use counts per bin per second so that assumptions about the sample volume dependence on maximum dimension don't affect the results).
+    - Program: Number_Distribution.py
+        - To run on individual file: python3 Number_Distribution.py filename
+        - Generated file looks like: Output_Files/Step4/20100101_HVPS3_columns_notwater_step4.pbp.nc
+    - View/plot with, ncview filename
+    - ncplot and ncdump should also work.
+    - Work is in progress to make matplotlib plots of the distributions for each second.