Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'NoneType' object has no attribute 'predict' - possible multiprocessing problem - Windows 10?? #124

Open
bluetyson opened this issue Sep 12, 2020 · 1 comment

Comments

@bluetyson
Copy link
Contributor

bluetyson commented Sep 12, 2020

The problem occurs when doing OOS validation and multiprocessing by the looks. A .model file is created and looks fine when I check it... as in, I loaded one created and it had right number of feature importances, not tried prediction. Same process/data worked on out of sample when not run multiprocessing. Model exists when checked in predict.py - kwargs and RandomForestTransformed get returned ok - and that branch clearly works fine in general - I have run quite a lot of multiprocessing regressions with no OOS section with no problem.

+59s INFO uncoverml.scripts.learn_cli [P0]: Learning full randomforest model
+83s INFO uncoverml.validate [P0]: Performing out-of-sample validation with 5301 targets...
Uncaught exception on processor 2
Traceback (most recent call last):
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\rscott\AppData\Local\Continuum\anaconda3\envs\gdaltest\Scripts\uncoverml.exe\__main__.py", line 7, in <module>
    sys.exit(cli())
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)


  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\uncoverml\scripts\__init__.py", line 142, in learn
    learn_cli.main(config_file, partitions)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\uncoverml\scripts\learn_cli.py", line 64, in main
    oos_results = ls.validate.out_of_sample_validation(model, oos_targets, oos_features, config)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\uncoverml\validate.py", line 349, in out_of_sample_validation
    lon_lat=pos)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\uncoverml\predict.py", line 56, in predict
    result = apply_masked(pred, data)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\uncoverml\models.py", line 1113, in apply_masked
    res = func(mr.trim_mask(data), *args, **kwargs)
  File "c:\users\rscott\appdata\local\continuum\anaconda3\envs\gdaltest\lib\site-packages\uncoverml\predict.py", line 41, in pred
    predres = np.reshape(model.predict(X, **kwargs), newshape=(len(X), 1))
AttributeError: 'NoneType' object has no attribute 'predict'
@bluetyson
Copy link
Contributor Author

bluetyson commented Sep 12, 2020

Basic config when encountered the problem :-

mpiexec -n 3 uncoverml learn N032Dummy.yaml

[same error if a percentage OOS or a separate shapefile]

learning:
  algorithm: randomforest
  arguments:
    n_estimators: 50
    target_transform: standardise
    random_state: 42

validation:
  out_of_sample:
    percentage: 0.2
    shapefile: smalloutofsample.shp
    property: target
  k-fold:
    folds: 3
    parallel: True
    random_seed: 42

features:
  - type: ordinal
    files:
    - directory: F:\Modelling
    transforms:
      - centre
      - standardise
    imputation: mean

targets:
    file: F:/targetsmain.shp
    property: target

mask:
  file: F:/mask.tif
  retain: 1
  
prediction:
  quantiles: 0.95
  outbands: 4

output:
  #plot_feature_ranks: True
  #plot_intersection: True
  #plot_real_vs_pred: True
  #plot_correlation: True
  #plot_target_scaling: True
  directory: F:/RESULTS/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant