=============================================================================== SWICS/SWIMS Level 2 Release Notes, Data Version 3.3 Jim Raines, Sue Lepri, and Thomas Zurbuchen =============================================================================== Overview -------- This SWICS/SWIMS Level 2 Version 2.0 represented an entirely new processing method allowing the extraction of a wealth of compositional data from the SWICS and SWIMS instruments. From this method, density, bulk velocity and thermal velocity are produced for over 60 ions, and assembled into numerous composite data products. This method, these data products, and the process of calculating them and assessing their quality are explained below. Version 3 contains several noticeable improvements over version 2 and is the culmination ofyears of analysis and interpretation. Method ------ Although SWICS has high mass, charge and energy resolution, considerable overlap exists among ions with similar mass or m/q. This overlap precludes direct assignment of individual counts to specific ions with a high degree of accuracy. To remove this overlap and perform this assignment, the SWICS raw ion events (so called Pulse height Analysis (PHA) words) are passed through a multi-stage inversion algorithm that allows individual measured counts to be assigned to specific ions in a way that preserves the Gaussian measurement statistics. Once assigned to one of over 60 individual ions in the analysis, these counts are converted to a set of phase-space densities as a function of velocity. Density, bulk velocity, and thermal velocity are then calculated by taking moments of these phase-space density distribution functions. This process is repeated for each time interval depending on the selected time resolution. (See von Steiger et. al., 2000, and Raines et. al., 2005, and references therein for further details This analysis produces a large amount of data, over 10,000 data items per timestep at a given time resolution. As a result, only selected combinations, those of greatest interest to the community and most reliably made on a production basis, are included in this dataset. In all cases, submitted data are based only on three moments of all distributions: density, radial speed, and radial temperature or thermal velocity. Ions Analyzed: -------------- The following ions are present in the analysis. Sums of ions, e.g. for elemental abundances or average charge states, include only these species. H: 1+ He: 1-2+ C: 4-6+ N: 5-7+ O: 5-8+ Ne: 6-9+ Mg: 6-12+ Si: 6-12+ S: 6-14+ Fe: 6-20+ Data Quality ------------ The quality of the moments are assessed in several ways: - the density error propagated from the counts in the phase-space density distribution function - comparison of thermal velocity to that calculated by fitting a Gaussian function to the distribution function - manual review and editing for each time-period The results of this process are encoded in the data set via a quality flag (qf) for each data item and, where appropriate, a fill value of -1.0. This quality flag is a bit-field, meaning that it is composed of several individual flags which can be set in various combinations, detailed below: Value Description ----- ----------------------------------------------------------------- 0 Good quality 1 Error in density measurements indicates low Statistics 2 Error in density exceeds allowable limit so that the data are not usable (and set to -1.0) 4 Distribution is non-thermal because it exceeds allowable limit when compared with a Gaussian fit of the phase space-density distribution function. Thermal velocity set to -1.0; 5 both qf=1 and qf=4 set (low stats. and non-thermal) 6 Both qf=2 and qf=4 set (very low stats. and non-thermal) 8 Insufficient data for construction of distribution function. Data is set to -1.0 Data deemed to be of low quality upon manual review by the SWICS/SWIMS instrument team are filled with -1.0. The quality flag is ignored under this circumstance. Note: Quality flags of 2 and 8 are functionally equivalent -- the data is filled to -1.0 -- though they stem from different sources. Data Products in Detail ----------------------- Time -- The timestamp for a given data point represents the beginning of the accumulation time and is derived directly from the instrument cycle time. 4He2+ density -- is derived from the zeroth moment (integral) of the distribution function, as defined by Von Steiger et al., 2000. Bulk velocities for 4He2+, C5+, O6+ and Fe10+ -- are derived from the first moment (center) of the phase-space density distribution function for any specified ion. Thermal velocities (vth) for 4He2+, C5+, O6+ and Fe10+ -- are the radial (1D) component of thermal velocities, representing the 2nd moment (width) of the phase-space distribution function for the specified ion. Since F10+ is one of very many states of Fe, it is not always a major component of the Fe charge state distribution and is thus subject to regular dropouts. Average charges -- are calculated as a density-weighted average of the charge for the available states for analyzed ions of the specified element. For oxygen and carbon, the a priori knowledge is applied to the calculation to improve stability. Average charge is only calculated when both O6+ and O7+ are present for oxygen. For carbon, C5+ and one of either C6+ or C4+ must be present. Because Fe has so many possible states with highly variable densities, two Fe states are required to calculate an average charge. These two charge states may not be adjacent; For detailed studies of ions with a large range of charge states, refer to the full charge state distributions, available at two hour resolution. Elemental abundances -- are the sum of the density of all available charge states relative to the total oxygen density. If oxygen is marked as low statistics, these values are considered unreliable and filled with -1.0. Full charge distributions -- give the abundance of each charge state relative to the total abundance for analyzed ions of the specified element. No restrictions are made on the number of states available, i.e. if only one good state is available the relative abundance of that state will be 1.0. Charge state ratios -- represent selected values from a full charge state distribution. As such, there are times when they are not a good indicator of conditions. For example, in cold solar wind, when O5+ and O6+ dominate, the O7+/O6+ ratio may either be very low or beyond allowable limits and filled to -1.0. Solar wind type -- represents a rough classification of solar wind type based on functions of O7+/O6+ versus proton speed as described by Zhou (2008). This parameter ranges from 0-2, where the values have the following meanings: Value Description ----- --------------------------- 0 Streamer Wind 1 Coronal Hole Wind 2 Coronal Mass Ejection (CME) Usage Caveats and Issues ------------------------ The efficiencies for these instruments are good to within 10-15%. These efficiencies are a component of the phase-space density distribution functions and, as such, are fundamental components of each value reported in this dataset. This overall uncertainty must be considered in any analysis. This method may underestimate 4He2+ density and thermal velocity during periods of very high flux (e.g. the Halloween Storm of 2003 (2003302-2003306)). In a number of the data series, values are very often marked with the low statistics flag. Despite this fact, most of these points are quite reliable. In consideration of the low statistics, conclusions in these series should not be based on less than 3 contiguous data points. For average charge states, care should be taken when integer values appear, i.e. = 4.0 for carbon. This indicates that only one charge state remained after inversion and error filtering. In many cases it does represent, to one significant figure accuracy, the of the plasma, but not in all cases. This condition often occurs when the solar wind density is very low and/or the speed and thermal speed are very low. Our current data validation methods do not catch every exception, and care should be taken when using these points, especially if they depart significantly from the neighboring points. In May 2000, the post-acceleration voltage of SWICS was increased to improve the resolution of time-of-flight and energy of the instrument. Increased separation of ion peaks in the tof-energy space resulted, as expected. However, quite unexpectedly, this produced a noticeable shift in the density of a number of ions. This change contributed to a change in several of these data series. So far we have identified noticeable changes in C/O, Ne/O, Mg/O, S/O, Ne charge state distributions, Mg charge state distributions, the O8+ component of the O charge state distributions. After substantial investigation, it was determined that this jump is caused by 1 more ions in the above species being resolved much better after the voltage increase. In the extreme cases of Mg9+ and Si10+, ions went from being nearly on top of each other to resolved fairly well. The inversion method was not designed to separate peaks in near-complete overlap, so it does not entirely compensate for this change and the jump results. Bottom line: The post-May 2000 data is the higher quality. Studies which do not cross the May 2000 boundry should not be affected by this issue. In particular, before May 2000, some crosstalk between charge states of sulfur and Fe9-11+ has been observed in the inversion. This results in an average charge state in this period which is approximately 10% higher than in data after May 2000. Several quantities appear to depend somewhat on the ACE spacecraft orbit and sensor viewing geometry, with a period of about 6 months. Most affected are Ne/O, Mg/O and 4He2+ density. We have developed an empirical correction which significantly reduces this effect. The correction method is uniformly applied to all recovered ion densities, from which quantities in this dataset are derived. It has been found that the 4He2+ density does not always match that of SWEPAM, but that the deviation is not clearly correlated with any quantities tested thus far. This behavior is still under investigation. A recent and comprehensive analysis of Ulysses SWICS data over its entire lifetime (von Steiger and Zurbuchen, 2011) provided a detailed and systematic analysis of statistical and systematic errors of solar wind heavy ions. There are important species-dependent errors that are being addressed. Due to the inherent similarity of the SWICS instruments of ACE and Ulysses, these uncertainties should apply here as well. In addition, recent and hitherto unpublished results of synthetic data testing indicate that errors in some ions may be larger than originally estimated. This new method uses realistic discrete count distributions, derived from monte-carlo sampling with estimated ion-specific statistical and systematic uncertainties. These synthetic data distributions are processed just as Level 2 data and can be compared with the input parameters. This process is repeated for a statistically-significant number of random 2 hour intervals to yield statistical estimates of recovered percentage and standard deviation of that recovered percentage. These estimates therefore provide information about uncertainty of the inversion process with the limitation of real data, both in systematic offsets and random variability (standard deviation of recovered percentage). This study is not yet complete. However, initial results, together with the already published analysis, allow us to identify ions that have large retrieval errors. Overall, ratios with larger errors are related to some N, Ne, Mg and Si ions. But, we also note that there are impacts to rare ionic charge states of O and Fe, which can also be adversely affected in this inversion process. The SWICS team is committed to raising this data to the highest quality possible and to improve the accuracy and usability of this data. All issues are taken seriously and investigated as resources allow. A discussion forum is maintained to provide more details on this process. Send mail to Jim Raines (jraines@umich.edu) if you wish to be added. Summary of Changes in Version 3 ------------------------------- v3.0: - Improved stability of count assignment to individual ions in inversion. - Improved accuracy of He/O ratio and He density, including substantial removal of orbit dependence. - Fixed bug in elemental abundance error flags; many fewer time steps now marked with 'low statistics' flag. - Added solar wind type indicator (Zhao et al. 2008) - Added elemental abundances at native 1 day resolution - Reduced data drop-outs due to processing code exceptions. v3.2: - Removed / greatly-reduced 6 month periodicity in elemental abundances. v3.3: - Corrected issue which caused Fe-containing data products to differ substantially between 2h and 1d resolution, evident mainly in long (~30 day) averages. Fe/O ratio changed by up to 50% in portions of both 2h and 1d datasets (unchanged in other portions). - Added 30 day average comparison to regular validation plots to catch any possible recurrence of similar issues. To Be Included in Future Releases --------------------------------- There are a number of extensions of submitted data that remain for future releases. We also intend to add additional data items to these releases based on community requests. Currently the following series are still not in full production: 3He/4He and Mg, Ne isotope ratios. References ---------- Raines, et. al., Heavy Ions in the Solar Wind: A New Dataset from ACE, Proceedings of the Solar Wind 11/SOHO 16, "Connectin Sun and Heliosphere" Conference (ESA SP-592), 12-17 June 2005 Whistler, Canada. Ed: B. Fleck, T.H. Zurbuchen, H.Lacoste. Published on CDROM. p.101.1 von Steiger, et. al., Composition of quasi-stationary solar wind flows from the Ulysses/Solar Wind Ion Composition Spectrometer, J. Geophys. Res., Vol. 105(A12), 27217-27238, 2000. Von Steiger, R. and Zurbuchen, T.H., Polar coronal holes during the past solar cycle: Ulysses obsevations, J. Geophys. Res, VOL. 116, A01105, doi:10.1029/2010JA015835, 2011. Zhao, L., Zurbuchen, T. H., and Fisk, L. A.. On the Global Distribution of the solar wind during solar cycle 23: ACE observations. Geophys Res Let 36, L14104, doi:10.1029/2009GL039181, 2009. Revision History: ----------------- Rev Date Authors Description 15Mar2008 JMR,STL Initial adaptation for version 3 data. A 18Apr2008 JMR Revised elemental abundance caveats B 17Mar2009 JMR,STL Added caveat involving Fe & S crosstalk in pre-rampup period. C 25Sep2009 JMR,STL Updated for version 3.2. D 02Oct2009 JMR,STL Added list of ions analyzed. E 20Apr2012 JMR,STL,THZ Added description of Fe-related bug. Added error description from synthetic data testing.