Error with QAP Build #87
Comments
|
you need the ConfigParser module:
pip install ConfigParser
…----------------------------------------------------------------
Alexander Li Cohen, M.D., Ph.D.
E-mail: [email protected] (Medical/Science Email)
E-mail: [email protected] (Lifetime Email)
----------------------------------------------------------------
On Tue, Jan 24, 2017 at 11:38 AM, TDeramus ***@***.***> wrote:
Hi.
I recently installed QAP on my local machine and on our cluster to process
ABIDE-II data.
However, when I attempt to run the pipelines, I get similar errors on both
environments (see attached photos below).
I'm not sure what is is that is going wrong here and I was hoping to get a
little input.
There's a module load error with the cluster I think, but I can confirm
that the local machine (bottom two photos) is running nipype verson 0.12.1
so I don't think that is the issue.
Happy to send the yml files if needed, but the configs are the generic
abide ones.
[image: clustertemporalerror]
<https://cloud.githubusercontent.com/assets/23127333/22256325/0338960c-e220-11e6-9a85-daff50cf671f.png>
[image: clusterspatialerror]
<https://cloud.githubusercontent.com/assets/23127333/22256326/033cb8cc-e220-11e6-8087-6dbab5010b0e.png>
[image: qaplocalanatomical]
<https://cloud.githubusercontent.com/assets/23127333/22256327/033f8be2-e220-11e6-82d2-5f97d95b7bf3.png>
[image: qaplocaltemporal]
<https://cloud.githubusercontent.com/assets/23127333/22256328/0341487e-e220-11e6-88ad-01d1144f4841.png>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#87>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIPa_1--vpAC43mQPqp1HEYoqI9_iG3Xks5rVikTgaJpZM4Lsfek>
.
|
|
Alternatively (for the configparser issue) you can use an earlier version of Python's future library by running pip install future==0.15.2 (see here). For the other issue, are there any crash files ending in '.pklz' being produced in the output directory? If so, could you please send them to [email protected] so that I can take a look at them? |
|
Is that specific future library supposed to be part of QAP's dependencies?
(checking the BIDS-App Docker, 0.16.0 is being installed as a dependency
for another package)
…-Alex
----------------------------------------------------------------
Alexander Li Cohen, M.D., Ph.D.
E-mail: [email protected] (Medical/Science Email)
E-mail: [email protected] (Lifetime Email)
----------------------------------------------------------------
On Tue, Jan 24, 2017 at 12:32 PM, John Pellman ***@***.***> wrote:
Alternatively (for the configparser issue) you can use an earlier version
of Python's future library by running pip install future==0.15.2 (see here
<nipy/nipype#1699>).
For the other issue, are there any crash files ending in '.pklz' being
produced in the output directory? If so, could you please send them to
***@***.*** so that I can take a look at them?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#87 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIPa_wxLgSgBQhzlrQ8J4QZ6HtQPOcUeks5rVjWWgaJpZM4Lsfek>
.
|
|
It's a dependency of nipype. QAP itself does not use future. |
|
Thank you for your input. Upon working on it some more, it looks like we're having library issues on the cluster (I still get error messages when working on the cluster after getting the dependency), I'm working with our resident software engineer to figure out what may be missing library wise and a few ways to fix it before I post again. However, I have been having some issues with qap_functional_temporal.py and qap_anatomical_spatial.py and the report function. Whenever I set write report to True, I get the following error: I think that may also be a missing library issue but I'm not sure. qap_functional_spatial.py will run to completion as long as this is not flagged however. qap_anatomical_spatial.py will not run to completion but that seems to be a memory issues (which should be fixed once I have it working on the cluster). qap_functional_temporal.py on the other hand, will throw the following error whenever I run it with reports set to false: I am not sure what is causing that. I'm sending the pklz files from the steps to the email you specified, but I am unsure that the core-dump ones contain any useful information. Let me know if you have any ideas, they would be appreciated. |
|
Hi @hermeckerogg , What happens if you open a Python terminal and type the following code? |
|
I get the following error when I run import numpy:
I will try installing both of those and testing it again. See what happens. Strange though because I could have sworn I had those. |
|
Hi @hermeckerogg , Those packages shouldn't be necessary for testing Numpy. You need to open a Python terminal by typing |
|
Doing that throws the following error:
|
|
Right. So, took a day to rebuild my python libraries and Conda setup and check dependencies. Those seem to be a little less bad now than they were. However, I am not getting the following error on every command I try to run: Any idea what might be causing that? |
|
The newest version of nipype is incompatible with QAP (see here). That's the most likely cause of this error. To downgrade use the following command: |
|
Alright. Think I found what the issue might have been. So the version of Nipype I had installed was 0.12.1, but I did it through Conda, which I've noticed does a good bit of dependency upgrade/downgrade shuffling here and there to make sure everything works. After removing it through Conda, removing any leftover bits through ruthless rm -rf thuggery and reinstalling via pip (as you describe above) it seems to be working. Need to try it with functional spatial, anatomical spatial, and with report writing options (since I also installed pygraphviz via conda) to see if they work, but maybe don't use the conda nipype packages. |
|
The code runs just fine, but unfortunately the summary report is producing an error. This is with write_all_outputs: False and write_report: True 170202-02:45:10,911 workflow INFO: |
|
Hi @hermeckerogg , This is an issue currently present in the pip-installed version of QAP. Please see here for a discussion about how to fix it. If the solution discussed there doesn't work, let us know so that we can troubleshoot further. Best, |
|
So I've actually tried that one a couple of times. Upon doing so after the dependencies were fixed, it returned the following: 170203-03:31:25,482 workflow INFO: 170203-03:31:25,531 workflow INFO: File "/home/Thomas/anaconda2/lib/python2.7/site-packages/qap-1.0.5-py2.7.egg/qap/cli.py", line 424, in _run_workflow File "/home/Thomas/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/workflows.py", line 597, in run File "/home/Thomas/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/linear.py", line 57, in run File "/home/Thomas/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/base.py", line 95, in report_nodes_not_run RuntimeError: Workflow did not execute cleanly. Check log for details 170203-03:31:25,673 workflow INFO: File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 1499, in run_script File "/home/Thomas/anaconda2/lib/python2.7/site-packages/qap-1.0.5-py2.7.egg/EGG-INFO/scripts/qap_functional_spatial.py", line 10, in This is with: write_all_outputs: False write_report: True This time it won't write the CSV file with the values in them. But it did produce error files which I'm zipping and sending now. |
|
Hi @hermeckerogg , The crash files you sent me implicate a step that occurs before report generation. In particular, the functionality for determining the GSR is failing. Could you run the following command and send me the output to take look at?
|
|
Appreciate the quick response. Unfortunately that last run did not produce any output files in the results folder. The last working run I had gave me this: Dataset File: rest_calc_resample_volreg_mask.nii.gz ----- HISTORY ----- I will try running it with the git build again with write_all_outputs set to True and see if that turns up anything. |
|
Hi @hermeckerogg , Sorry- I neglected to consider that write_all_outputs was set to False. I have another question- which version of NumPy are you using? Starting with version 1.12 of NumPy, the ability to slice arrays using non-integers has been dropped (I'm going to leave a link to the changes here for future reference), which is probably what is throwing off the GSR function. |
|
Ok so with the new flags the code rand to completion and the output for 3dinfo is: Thomas@DESKTOP-PNO0KIE:/mnt/f/ABIDEII/ABIDEII-SDSU_1$ 3dinfo /mnt/f/ABIDEII/ABIDEII-SDSU_1/QAP_Functional_Temporal_working_funct/28878/session_1/rest_1/func_get_brain_mask/rest_calc_resample_volreg_mask.nii.gz Dataset File: /mnt/f/ABIDEII/ABIDEII-SDSU_1/QAP_Functional_Temporal_working_funct/28878/session_1/rest_1/func_get_brain_mask/rest_calc_resample_volreg_mask.nii.gz ----- HISTORY ----- The important part seems to have worked. However, when it gets to the stage of making the graphs and writing the PDFs, there seems to be an error. /home/Thomas/anaconda2/lib/python2.7/site-packages/qap-1.0.5-py2.7.egg/qap/viz/reports.py:39: FutureWarning: sort(columns=....) is deprecated, use sort_values(by=.....) File "build/bdist.linux-x86_64/egg/pkg_resources/init.py", line 1499, in run_script File "/home/Thomas/anaconda2/lib/python2.7/site-packages/qap-1.0.5-py2.7.egg/EGG-INFO/scripts/qap_functional_temporal.py", line 10, in It will produce some PDF files, but the data inside of it is minuscule (attached). |
|
Also according to the output I'm using: So that may answer that. I can try a rollback and see what that does. |
|
@hermeckerogg Quick heads up- QAP 1.0.8 was released (available via If you can, give it another shot and let us know if you are still running into trouble. |
|
does 1.0.8 allow for multiprocessor runs yet?
(I'm almost done re-working the BIDS-app to use the new sublist generator
and qap_measures_pipeline.py, testing now...)
…-Alex
----------------------------------------------------------------
Alexander Li Cohen, M.D., Ph.D.
E-mail: [email protected] (Medical/Science Email)
E-mail: [email protected] (Lifetime Email)
----------------------------------------------------------------
On Tue, Feb 28, 2017 at 3:11 PM, sgiavasis ***@***.***> wrote:
Quick heads up- QAP 1.0.8 was released (available via pip install qap or
here: https://github.com/preprocessed-connectomes-
project/quality-assessment-protocol/releases/tag/v1.0.8).
If you can, give it another shot and let us know if you are still running
into trouble.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#87 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIPa_xFksTQjhzV7sL7vctCwo4XJtxhBks5rhH9xgaJpZM4Lsfek>
.
|
|
Yes, definitely. Multi-processor, cluster runs (SGE, PBS, SLURM), AWS compatibility, ..etc. You can configure your resource settings in the pipeline configuration YAML: |
Hi.
I recently installed QAP on my local machine and on our cluster to process ABIDE-II data.
However, when I attempt to run the pipelines, I get similar errors on both environments (see attached photos below).
I'm not sure what is is that is going wrong here and I was hoping to get a little input.
There's a module load error with the cluster I think, but I can confirm that the local machine (bottom two photos) is running nipype verson 0.12.1 so I don't think that is the issue.
Happy to send the yml files if needed, but the configs are the generic abide ones.
The text was updated successfully, but these errors were encountered: