Comments (9)
2.5 mil is quite a lot more than what I tested it with (up to 10k). But it tells you that 1722860 genes are empty, so you're actually "just" testing 800k genes.
I would recommend you to:
- To test run time, set R=1 and subset to a few methods by setting the tests argument, e.g., just the simple t-test and wilcox test; tests = c("ttt","wil").
- Use more cores if you can, Using only 1 core significantly increases run time
- Note that your predictor is assumed to be quantitative, so you should write it as factor(c(0,1,0,1,0,1,0,1)) or c("A","B","A","B","A","B","A","B").
From extrapolation I would say it should take around 40 min to run t-test and wilcox test on 800k genes with R=1 and with 1 core. Increase it to 2 cores and the run time is about half.
When you have tested that it can run within an acceptable time, you can add other tests. If its RNAseq data try adding edgeR ("ere","ere2","erq","erq2")
Cheers
from datest.
Hi again,
I have added a new function to estimate runtime of the different methods; runtimeDA.
This will tell you whether some methods are simply too slow on your dataset. You can then use the tests argument to specify the methods that you want to include.
Cheers,
Jakob
from datest.
Thanks :).
Right now I'm busy with other things, so I'm letting it run, but it seems that it's indeed a bit slow for this size of data.
It's still running and producing output, as it seems (started at the 4th, last output written yesterday morning), and I anyways need to test it on the whole dataset (well, it's actually not the whole, as you noted).
(if it doesn't stop within the next week, I'll try with more cores; just need to monitor the RAM usage, to get an estimate)
from datest.
FYI: It ran 28 days on 10 cores, with a maximum of 112GB RAM.
And then I made an error in my R script and the whole thing aborted, stupid me.
I have the plot with AUC and FPR. I guess that is enough to make a conclusion, right?
(AUC is pretty low though, highest 0.6)
from datest.
Wow, well at least it finished. AUC of 0.6 is a bit low. You might wane to spike some more features when the dataset is that large. Also, you can subset to only fast methods. This for example:
mytest <- testDA(x, as.factor(c(0,1,0,1,0,1,0,1)), effectSize = 4, k = c(50,50,50), tests = c("ttt","ltt","ltt2","wil","ere","ere2","lim","vli","lli","lli2"), cores = 10)
This will spike 150 features with effect size of 4, and test with 10 fast and relevant methods.
That being said, if you have the plot it should be fine. So long as it treated your predictor as categorical. If the code is as in your first post it would be wrong, you should wrap the predictor in as.factor().
Also, if you run it again, install DAtest again to get the latest version.
Cheers
from datest.
Thanks :). I changed the predictor before I ran the test, because of what you said.
I installed now also the latest version.
If I try to run the command you put here, I get an error though :/.
The output is:
mytest <- testDA(x, c("A","B","A","B","A","B","A","B"), effectSize = 4, k = c(50,50,50), tests = c("ttt","ltt","ltt2","wil","ere","ere2","lim","vli","lli","lli2"), cores = 20)
You are about to run testDA using 20 cores. Enter y to proceed
Error in testDA(x, c("A", "B", "A", "B", "A", "B", "A", "B"), effectSize = 4, :
Process aborted
Execution halted
My R is not brilliant, but I can tell at least that the syntax itself is right, and I also checked in testDA.Rd, there don't seem to be any issues.
But where is the error? Is probably a general R thingy, which I cannot interpret right now.
from datest.
Ah, it's because you have to enter "y" (without quotation marks) after you run the testDA function:
You are about to run testDA using 20 cores. Enter y to proceed
When you're running with many cores (>10), you have to confirm the run. It's just an extra check, to ensure that a server is not accidentally overloaded with parallel workers.
from datest.
Aaaah, thanks.
Okay, had it running with 10 cores, because I couldn't be bothered to figure out how to pass a "y" into the non-interactive session.
Has now finished. The AUC is still max 0.6, and since I now also got the table, I see that the spike in detection rate is constantly 0, except for 1 case, where it's 0.03333.
So...I guess I'd need both more spike ins and a bigger effect size....right?
from datest.
You can just run like this:
mytest <- testDA(...); y
You should not expect to get very high AUC or spike.detect.rate for this size of data. You can try k=c(500,500,500) and effectSize = 5. As long as the spike.detect.rate is above zero for the method that has FPR < 0.05 and the highest AUC I would go with that.
from datest.
Related Issues (20)
- min.sample argument in preDA function HOT 1
- Errors in Mac Installation HOT 2
- Adding interaction terms in DA.neb() HOT 4
- Parallelization does not work in R 4.0 + Rstudio HOT 4
- Clarification on terminology of "raw counts" vs. "absolute abundance" HOT 2
- Preform comparisons on rank (i.e. family) level instead of OTU HOT 2
- Add corncob
- Cannot install HOT 2
- Phylum level - warning HOT 1
- Addition of ANCOM and ANCOM-BC HOT 4
- Error in rowSums HOT 6
- Implement UpsetDA
- Posthoc for aov with predictor and covar? HOT 1
- Getting error "subscript out of bounds" when trying to run testDA()
- Tweedieverse HOT 1
- adx was excluded due to failure HOT 5
- Best Practices for >2 treatments HOT 1
- DA.aov/DA.lao and DA.TukeyHSD HOT 4
- showing p-value on the graphs HOT 1
- Score = 0 for all tests and elaboration on running PCoA prior to DAtest HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datest.