EARTH-TIME.org's flagship data-processing and workflow automation software.
Home Page: http://cirdles.org/projects/et_redux/
License: Apache License 2.0
A repository for general issues for CIRDLES projects.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
We have been running our live data reduction for every sample for the past few weeks now and have found some things that could improve the output data. I will be creating a few issue posts as some are longer than others.
Our first issue goes as follows. Once we reach ~170 analysis the program begins an infinite loop of loading/number crunching. Redux performs so much math that we are not able to reject fractions or change filters. It behaves as if the program is frozen but will still refresh with time. We see the grey bar progress to 99% then add that fraction as well as refresh all the views, but then redux immediately starts to reduce the next fraction. This doesn't allow us to see what has just been reduced let alone change sliders or zoom in on concordia or the PDP. We also are unable to remove any unwanted fractions because of this incredibly long loading time. With that said, this behavior makes redux fall significantly behind the mass spec. We need the data to import smoother or in a more reasonable amount of time. Possible solutions that I have are to break long runs into distinct chunks, or run the reduction in the background, and have a manual button to import or show the data that had been reduced in the background.
As an example of breaking up the run into chunks or segments: for a run of n=300 would it be possible to break the whole run into thirds. This would allow us to use the reduced data from the first 100 while the next 100 is being reduced. Another possibility would be similar to Gehrels’ agecalc program. Agecalc uses a continuous running six average to reduce the data, and leaves data outside this window to not be recalculated.
My second idea would be to pause of hide the automatic refresh. For example, if we are 100 analysis in and looking at a population and applying discordant or reverse discordant filters and selecting our best age, we need time to do this, but right now the constant loading and refreshing of views is impossible to get around. So, if we could pause the refreshing of new data, but continue to have the data being compiled and reduced in the background we would be able to use the plots more efficiently and look closer at the data. Once we are done looking at what we wanted, we would then be able to refresh the live view and insert the data that has been reducing in the background in to the live data view and continue running the program as normal.
I understand this is a lot for one issue, but this has to be fixed for redux because we aren't able to use the live data and obstructs the usability of redux especially for users with large-n detrital samples. I hope the ideas help and let me know what you all are thinking.
I am getting an error message when I try to open PROBABILITY-DENSITY.svg files created by redux in Illustrator. The error reads "The operation cannot be completed because of an unknown error. [CANT]" To test if this was a general problem, I generated a svg file of a concordia plot, and I was able to open this in Illustrator without any problem. It seems to be a problem with the files being created for the probability density plots. I am running Redux 3.6.6 and IllustratorCS5.1.
I think it would be useful to remember the settings when loading a sample. This would save literally milliseconds but would be very useful once there are more labs involved. It could get messy and prone to mistakes if we have to search a specific lab each time. I already noticed the first pull down window in the loading page is saved. Nice touch. If all three could be remembered that would be swell.
Since GSA has came and went I have a few suggestions to make the workflow a bit easier. These ideas are for cosmetic and overall ease of use.
Open the 'Project Manager' or process new raw data to bring up the 'Project Manager' window, then change the role of one or more of the 'Unknowns' to 'Secondary RMs'. The dropdown menu reads 'Secondary RM' until you hit 'Save Project', when the roles spontaneously switch back to 'Unknown'. The role should stay as 'Secondary RM'.
Quick workaround is to open the sample date interpretation manager and click the probability tab, then the clear filters button.
This corresponds to the loading time issue we have been dealing with. When a new fraction is added to the live data table it goes to the bottom of its designated column. This should continue, but we would like there to be a window that shows us the name of the last acquisition along with the 6/8, 7/5, and 7/6 ages and uncertainties. This would be helpful when redux begins its infinite loop when we can't even scroll down to the data table. If the ages and uncertainties were added to a box under the concordia so we can see them with out scrolling down that would be especially useful.
When creating a new legacy data entry and trying to upload to sesar/geochron there is a problem with getting an IGSN.
I think that if we have the same option here as we do when we are putting data up to geochron after we reduce it. This would make this process much smoother.
So, if we can get a field to validate our geopass and then have sesar make an IGSN for our legacy data
Hi,
As I have been working with the data table and dealing with poorly behaved standards I have haven't figured out how to remove the standard from just one type of fractionation whether its 6/8 or 6/7. When I remove that fraction it takes it out of both when there is only an issue with one (6/7 or 6/8). Is there any way to select that or choose which one I would like to take a standard out of?
I have tried multiple times to open redux version 3.6.9. I get to the home screen but when I select LA-ICPMS nothing happens. It doesn't crash, it just doesn't open....
When Pbc is >blank Pbc, one has the option of partitioning the Pbc subtraction from reservoirs with different compositions (blank + Stacey and Kramers, e.g.). However, if the sample is spiked with ET2535, this function does not work. While it is uncommon to need this (given that if Pbc is an issue, mass fractionation is not likely a big source of uncertainty), in working on a subset of very rare and very old zircons, it was a good idea to spike and analyze some leachates from chemical abrasion derived from step leaching single zircons. These leachates have a couple pg of Pbc instead of the usual and it would be nice to be able to test the sensitivity of those dates to the Pbc correction.
On the live data reduction we would like to be able to have a button to show or hide the fractionation plots. The fractionation plots are excellent for the minders and people familiar with u/pb geochronology, but for visitors and people learning for the first time it can get rather confusing. So, if we would be able to hide those plots when necessary and have the concordia and PDF plots take up the space instead.
Once again, the plots in the live reduction are very important to see whats going on so all we request is to be able to hide it when we don't need to be monitoring it.
-Dan
Hi Jim.
We have just upgraded to the new version of ET Redux. In the new version the top panel that contain the measured ratios has gone. Some of us find that panel useful so we can look at the measurement uncertainty and the sample/tracer ratio, and practically it helps check reduced data to measurement files (sometimes handy). Would it be possible to reinstate this? Noah said people found it distracting but it has some use for some of us, and it could be hidden now you have this drag-able divider?
Cheers,
Dan, Blair, Diana, and Simon.
When filters for discordance, reverse discordance, and uncertainties are set they are reset to the default values after every new grain. If they could lock after you set them so we don't have to keep adjusting them this would be great. Also if we could set a default filter for these before we begin the run.
Since v3.0.4 has been released to the public and is considered stable, it should be merged into master.
I would recommend the following (perhaps in a fresh clone):
git checkout mastergit pull [upstream master], updating mastergit fetch --tags [upstream], ensuring that the tag v3.0.4 is known locallygit rebase v3.0.4, merging the changes without creating a new commitgit push [upstream master]Note that the parameters in brackets are only used when working from a clone of a fork (and assume that upstream is a remote pointing to CIRDLES/ET_Redux).
Hi,
I'm having trouble generating weighted means. I haven't been able to do this for any of my data sets, I will send an email out with a Redux file.
Thanks!
Clay
All is going well with the newest version of redux for the E2 lab here at laserchron! Recent visitors have loved all the features corresponding to the live data reduction. Right now redux is only using FC to reduce the data. However, we use SL and R33 in agecalc as well. We are wondering if it would be possible to have these standards included in the fractionation plots as well?
Release 3.2.1 takes an initial pass at storing accepted parameter models as resources with the GJ1 mineral standard model. This enhancement requires designing and implementing a more general and robust solution.
I am running Redux 3.6.6. Aliquot manager windows will not open while Redux is in live workflow. This may be intentional, but it took me awhile to figure out why the windows were not opening. If it is necessary to stop live workflow when the aliquot manager window is open, I think it would be better to have this happen automatically rather than the aliquot manager window just not opening. When a user tries to open an aliquot manager window, this action could automatically stop live workflow.
New samples in the results table are remaining gray even after the initial data is input. Noah confirmed this behavior on his computer and e-mailed Jim an example redux file a couple weeks ago. I have since noticed that if I open and close the sample dates window, this fixes the problem and all fractions that contain data turn black.
When user tries to change the table to settings specified by a local Report Settings on their computer, nothing happens.
When we run redux Pb204 has a baseline with a value below zero. I have looked at what our actual backgrounds are and I am seeing that we are starting around ~300 cps. This is quite a bit different and will show more reverse discordance in older grains. Which is a trend we have recently been seeing. Hopefully we can get a fix to this as soon as possible. I know that may be complicated with the math but I can hope.
It would be nice to be able to do regressions on TW plots for LA-ICP-MS data, with the option to peg the upper end of the regression to a given 207Pb/206Pb.
While refitting individual functions for FC reference materials I was running into issues when I would try and manually adjust the y-intercept. It would move and adjust how I wanted, but as soon as I would go to the next FC and open the local fit functions the y-intercept for the very last FC I worked on reverted to its original value. If we could make it so the y-intercept after it is changed would lock that new value instead of changing every time a different analysis is updated. Would greatly make fitting functions less frustrating.
-DA
Live mode went to a blank screen around spot 360 on a large n run. This is the first I've seen it crash, but this was with 3.6.12 I will see if it was just a user error.
I just archived two aliquots from the same sample to geochron.org. In the 'Archive Aliquot to Database' window, the first time I went to upload the second aliquot, the following text was displayed in red: "Note: This aliquot exists in Geochron. You may update it by choosing 'overwrite'." I think Redux noticed that I had already archived the first aliquot with the same IGSN, but this message wasn't true since I hadn't uploaded the second aliquot yet. I chose 'Overwrite' and the upload was successful, but the message was confusing.
Dear Mr. Bowring, I'd like to know if ET_Redux is currently able to handle data from Thermo Finningan Neptune?
Felipe
During demo of live mode at Laserchron on WIndows machine, noted that sliders on report table not present. Table scrolls ok with mouse however.
When using the csv import for legacy data, I am getting an error message. I am using the MIT legacy csv template. I then go to new sample from legacy analysis>ID TIMS legacy analysis (MIT) and select the csv. When I do this, I get an error saying "Document error: Premature end of file". If I press okay, it then takes me to the usual csv import screen and still seems to work otherwise. I am running ET_Redux 3.6.6. I was not able to attach a csv to this report, but can e-mail one of the files.
This is pretty minor, but as I was pasting fraction weights into the 'manage sample' window, I noticed that the default Mac keyboard shortcut for pasting (command-v) doesn't work; instead it requires control-v. It looks like other keyboard shortcuts (select all, copy, cut) also require using 'control' instead of 'command'. Not sure if it is possible to have separate keyboard shortcuts for Mac or Windows in the same java program, but I wanted to check. Thanks!
Ryan Frazer
UNC-Chapel Hill
There is a bug with the upper intercept regressions when using ET_redux 3.0.4 to regress legacy data entered as a UCSB LASS project. The regressions work, but when I select "truncate regression curve below concordia" in the concordia plot display settings, the regression line disappears completely.
The attached .redux file appears to reproducibly open correctly in redux v2.7.55, but silently declines to open in v3.4
I was working in the raw data window today and was repetedly switching between fitting functions to individual reference materials and the measures intensities tabs. When I would do this the horizontal scroll to navigate the individual fractions would go to wherever I was on the previous tab. This is frustrating because each tab isn't perfectly lined up with all of the others, so when I'm trying to fit functions for FC-21 and I want to check the measured intensities or see the whole session plot when I switch to the desired tab it appears at the same scroll distance where I initially was. With that being said, yes it was confusing, if both scroll bars could be independent for each separate tab that would make the functionality of flipping between individual tabs much easier with in the raw data manager.
-DA
The pre-maven help system is no longer viable and references and threads for it should be removed. We-based help is in preparation by @baldwinjj .
Downloading a (randomly selected) aliquot from geochron (http://geochron.org/upstreamviewfile.php?pkey=7282) and loading it into Redux via Sample -> New Compilation of Aliquots into SuperSample File, then choosing the downloaded .xml file returns the following error:
When there is a new fraction added to the data table both the concordia and PDP are refreshed to add that new fraction. When the plots reset they automatically revert the positive and negative discordance filters back to 100%. This doesn't make sense as you only have a short amount of time to view the changes because of the next acquisition being added so soon. We would want the filters to stay at the values we give them until we change them again or turn them off. When a new fraction is added it will only be plotted if it corresponds to the set filter values. This will give us a much better visualization of data the data if the filters aren't constantly being reset.
The sample-by-sample viewing of unknowns, controlled by the dropdown list of samples at the top left of the Raw Data Manager window is pretty cool, and a great way to look through the data for secondary reference materials and unknowns that are all (roughly) the same age. However, locking the y-axes together for all of the unknowns, especially for a detrital sample, forces analyses with very different scales onto the same y-axis. The result is that it's difficult to see the data for any one of the analyses.
I observed this issue when moving a .redux file saved by U-Pb_Redux 2.7.55 on a front-end mass spec PC to ET_Redux 3.3.3 on a mac laptop, but it seems other users have experienced similar problems with other version combinations, so this issue may be more general.
As of now, our background index for calculating baseline is correct for our U/Pb analysis within redux. We would like to be able to change the index for which redux calculates this baseline in the parameters window before we start a run. This would aid us by letting us manually change which indices are selected for baseline without having to get a patch from Jim. Right now, our U/Pb method on the Element 2 is excellent and we don't expect any changes in the near future. Having a default setting, which would be what it is set to now, and then being able to edit and save new setting would be excellent. Thanks
-DA
Redux appears to be plotting a histogram incorrectly. The sample has 22 data points, but the histogram bars only account for 21 data points.
I cannot attach the redux file here, but will e-mail it (sample 06-12).
The problem occurs when I set the histogram bin width to 50. The plot appears to miss the data point with a date of 1059 Ma.
These are just a few simple cosmetic adjustments George has request for the live data reduction window.
Other than just those cosmetic issues both the fast uncertainties and full propagation are working incredibly. The speed of which the data comes in is incredible, and as of these past two weeks the fractional plots have been looking excellent. George has also noticed that the uncertainties in the live views are almost 4 times the uncertainties we get in agecalc, for all three different ages. I have to still work more with the actual data manager mode to see if I can bring those uncertainties closer to agecalc.
Thanks;
-Dan
Is there a possibility of allowing a 202-205-235 spike in ET_Redux (the spike we use in Oslo)?
Redux does not seem able to perform common Pb corrections on my data. I only register the 206/204 ratio in Tripoli, could that be the problem? Could you please advice me on which ratios to read?
Thanks in advance for your help.
As of 175db2f, there's a large test suite that's disabled out of the box in pom.xml.
After some prodding, it seems that there are a few issues preventing the test suite from being a useful verification tool:
<source class>Test, hampering IDE features (such as Netbean's Test File).ETException's constructor opens a dialog, forcing the developer to manually close several of them during automated testing.ETException and reenabling the test suite, three tests fail.While improving readability is a big task, the first four three points are not. I'll try and get things working in the coming week.
Update: While messing with the tests on another machine, I was unable to replicate the test failures that I originally reported.
There appears to be one or more bugs when calculating the weighted mean by sample in the Sample Data Interpretations window when live workflow is running. I have seen two different behaviors. The first is that when I try to calculate a weighted mean by sample for a sample with two different aliquots, Redux kept un-checking and excluding the final fraction from the second aliquot each time live workflow updated. Similarly, when I just tested this issue on my laptop, when I went to the weighted mean window, then clicked the "by sample tab", the aliquot fraction folder kept closing. I opened the folder to select what aliquots to include in the weighted mean calculation, but each time live workflow updated, it would close the folder.
When outputting a weighted mean to pdf place the fraction information from the left hand column in the file as well. That way all information is available to the user.
A quick grep reveals that there are currently 63 classes with main methods, only one of which is being used as an entry point for the program.
At a glance, many of these mains seem to be integration tests for object serializations. It would be straightforward and beneficial to extract these methods into JUnit tests, so that they could be run automatically.
For reference, below is the output of grep -rl 'void main' ., showing all files with a main method.
./src/main/java/org/earthtime/dataDictionaries/RatioNamePrettyPrinter.java
./src/main/java/org/earthtime/ETRedux.java
./src/main/java/org/earthtime/matrices/matrixModels/CorrelationMatrixModel.java
./src/main/java/org/earthtime/matrices/matrixModels/CovarianceMatrixModel.java
./src/main/java/org/earthtime/matrices/matrixModels/CovarianceMatrixWithSubMatricesModel.java
./src/main/java/org/earthtime/matrices/matrixModels/JacobianMatrixModel.java
./src/main/java/org/earthtime/physicalConstants/PhysicalConstants.java
./src/main/java/org/earthtime/ratioDataModels/initialPbModelsET/commonLeadLossCorrectionSchemes/CommonLeadLossCorrectionSchemeA1.java
./src/main/java/org/earthtime/ratioDataModels/initialPbModelsET/commonLeadLossCorrectionSchemes/CommonLeadLossCorrectionSchemeA2.java
./src/main/java/org/earthtime/ratioDataModels/initialPbModelsET/InitialPbModelET.java
./src/main/java/org/earthtime/ratioDataModels/mineralStandardModels/MineralStandardUPbModel.java
./src/main/java/org/earthtime/ratioDataModels/pbBlankICModels/PbBlankICModel.java
./src/main/java/org/earthtime/ratioDataModels/physicalConstantsModels/PhysicalConstantsModel.java
./src/main/java/org/earthtime/ratioDataModels/rareEarthElementsModels/RareEarthElementsModel.java
./src/main/java/org/earthtime/ratioDataModels/tracers/TracerUPbModel.java
./src/main/java/org/earthtime/ratioDataViews/MineralStandardUPbRatiosDataViewEditable.java
./src/main/java/org/earthtime/ratioDataViews/MineralStandardUPbRatiosDataViewNotEditable.java
./src/main/java/org/earthtime/ratioDataViews/PhysicalConstantsDataViewEditable.java
./src/main/java/org/earthtime/ratioDataViews/PhysicalConstantsDataViewNotEditable.java
./src/main/java/org/earthtime/ratioDataViews/RatiosDataViewEditable.java
./src/main/java/org/earthtime/ratioDataViews/RatiosDataViewNotEditable.java
./src/main/java/org/earthtime/ratioDataViews/TracerUPbRatiosDataViewEditable.java
./src/main/java/org/earthtime/ratioDataViews/TracerUPbRatiosDataViewNotEditable.java
./src/main/java/org/earthtime/UPb_Redux/aliquots/UPbReduxAliquot.java
./src/main/java/org/earthtime/UPb_Redux/customJTrees/CheckBoxNodeTreeSample.java
./src/main/java/org/earthtime/UPb_Redux/dateInterpretation/vermeeschKDE/FFT.java
./src/main/java/org/earthtime/UPb_Redux/dateInterpretation/vermeeschKDE/Tester.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/AboutBox.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/DialogEditor.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/fractionManagers/FractionNotesDialog.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/graphManagers/GraphAxesDialog.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/projectManagers/BasicDnD.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/projectManagers/ProjectManagerFor_LAICPMS_FromRawData.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/sampleManagers/GeochronSampleCustomMetadataDialog.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/sampleManagers/heatMapManagers/HeatMapManager.java
./src/main/java/org/earthtime/UPb_Redux/dialogs/sampleManagers/sampleDateInterpretationManagers/SampleDateInterpretationAny2AxesChooser.java
./src/main/java/org/earthtime/UPb_Redux/fractions/AnalysisFraction.java
./src/main/java/org/earthtime/UPb_Redux/fractions/Fraction.java
./src/main/java/org/earthtime/UPb_Redux/fractions/UPbReduxFractions/UPbFraction.java
./src/main/java/org/earthtime/UPb_Redux/initialPbModels/InitialPbModel.java
./src/main/java/org/earthtime/UPb_Redux/mineralStandardModels/MineralStandardModel.java
./src/main/java/org/earthtime/UPb_Redux/pbBlanks/PbBlank.java
./src/main/java/org/earthtime/UPb_Redux/reports/ReportSettings.java
./src/main/java/org/earthtime/UPb_Redux/samples/SampleMetaData.java
./src/main/java/org/earthtime/UPb_Redux/samples/SESARSampleMetadata.java
./src/main/java/org/earthtime/UPb_Redux/tracers/Tracer.java
./src/main/java/org/earthtime/UPb_Redux/utilities/BrowserControl.java
./src/main/java/org/earthtime/UPb_Redux/utilities/comparators/IntuitiveStringComparator.java
./src/main/java/org/earthtime/UPb_Redux/valueModelPanelViews/MineralStandardUPbRatiosPanelViewNotEditable.java
./src/main/java/org/earthtime/UPb_Redux/valueModelPanelViews/ValueModelsPanelViewEditable.java
./src/main/java/org/earthtime/UPb_Redux/valueModelPanelViews/ValueModelsPanelViewNotEditable.java
./src/main/java/org/earthtime/UPb_Redux/valueModels/MeasuredRatioModel.java
./src/main/java/org/earthtime/UPb_Redux/valueModels/MineralStandardUPbRatioModel.java
./src/main/java/org/earthtime/UPb_Redux/valueModels/SampleDateModel.java
./src/main/java/org/earthtime/UPb_Redux/valueModels/ValueModel.java
./src/main/java/org/earthtime/UPb_Redux/valueModels/ValueModelReferenced.java
./src/main/java/org/earthtime/UPb_Redux/valueModelViews/MineralStandardUPbRatioViewNotEditable.java
./src/main/java/org/earthtime/UPb_Redux/valueModelViews/ValueModelViewNotEditable.java
./src/main/java/org/earthtime/utilities/jamaHelpers/MatrixRemover.java
./src/main/java/org/earthtime/utilities/TimeToString.java
./src/main/java/org/earthtime/visualizationUtilities/agePicker/AgePickDemo.java
./src/main/java/org/earthtime/visualizationUtilities/ProgressBarViewer.java
./src/main/java/org/earthtime/xmlUtilities/SimpleTransform.java
I have noticed that there is a miss labeling for standards in live mode.
FC goes as follows: after FC-4 numbers start to be skipped and are only even
FC-1
FC-2
FC-3
FC-4
FC-6
FC-8
FC-10
FC-12
FC-14
etc,...
R33 and SL follow a similar trend but start at index two and then continue to skip odd numbers.
This seems to be just a minor labeling issue, but to visitors it may appear as an issue with standards
Hello!
The issue: as data is being pulled into Redux the uncertainties of 206/207 seem fine, but 206/238 age and errors fly through the roof the longer the acquisition continues. We witnessed an unknown of 180 Ma, and the very next aliquot, also an unknown, came in at 3.9 Ga. Subsequently, the aliquot prior (180 Ma) increased to 227 Ma, and uncertainties rose. It looks like the unknowns are 'correcting' each other.
Whenever a saved LA-ICPMS project is opened in Redux, the first time the user clicks 'Project Raw Data', the 'fractionation technique' defaults to 'Downhole', even if the sample was interpreted and saved with the 'Intercept' method. This choice is important and should be preserved through opening and closing the file. When opened for the first time, the 'Project Raw Data' window should default to the last saved choice for fractionation method and display that output.
Hi all,
I have some raw LA-ICPMS data I am trying to reduce, but for some reason ET_Redux will load it to 6% and then stop. I tried loading the dataset multiple times on different versions of Redux and every attempt failed at 6%. I've attached a .zip file containing the raw data in a separate email. I think it probably has something to do with the raw data file, however I can't find any obvious differences in this dataset and other functional ones.
Thanks,
Clay
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.