Forums

I have this problem.

I am executing the pycaret and the system return this mensage for me:

WARNING:cannot find psutil installation. memory not traceable. Install psutil using pip to enable memory logging.

deleted-user-7226569 | 11 posts | March 29, 2021, 10:13 p.m. | permalink

[formatted by admin]

Traceback (most recent call last):
  File "/home/marcogiampietro/jobs/robo.py", line 90, in <module>
    exp_clf101 = setup(data = dfObj, target = 'predict', session_id=123, silent=True)
  File "/home/marcogiampietro/.local/lib/python3.8/site-packages/pycaret/classification.py", line 580, in setup
    return pycaret.internal.tabular.setup(
  File "/home/marcogiampietro/.local/lib/python3.8/site-packages/pycaret/internal/tabular.py", line 1180, in setup
    display.move_progress()
  File "/home/marcogiampietro/.local/lib/python3.8/site-packages/pycaret/internal/Display.py", line 72, in move_progress
    self.progress.value += value
  File "/usr/lib/python3.8/site-packages/traitlets/traitlets.py", line 585, in __set__
    self.set(obj, value)
  File "/usr/lib/python3.8/site-packages/traitlets/traitlets.py", line 574, in set
    obj._notify_trait(self.name, old_value, new_value)
  File "/usr/lib/python3.8/site-packages/traitlets/traitlets.py", line 1134, in _notify_trait
    self.notify_change(Bunch(
  File "/usr/lib/python3.8/site-packages/ipywidgets/widgets/widget.py", line 367, in notify_change
    self.send_state(key=name)
  File "/usr/lib/python3.8/site-packages/ipywidgets/widgets/widget.py", line 263, in send_state
    self._send(msg, buffers=buffers)
  File "/usr/lib/python3.8/site-packages/ipywidgets/widgets/widget.py", line 487, in _send
    self.comm.send(data=msg, buffers=buffers)
  File "/usr/lib/python3.8/site-packages/ipykernel/comm/comm.py", line 122, in send
    self._publish_msg('comm_msg',
  File "/usr/lib/python3.8/site-packages/ipykernel/comm/comm.py", line 66, in _publish_msg
    self.kernel.session.send(self.kernel.iopub_socket, msg_type,
AttributeError: 'NoneType' object has no attribute 'session'
deleted-user-7226569 | 11 posts | March 29, 2021, 10:49 p.m. | permalink

[formatted by admin]

2021-03-29 22:33:48,937:INFO:PyCaret Supervised Module
2021-03-29 22:33:48,937:INFO:ML Usecase: classification
2021-03-29 22:33:48,937:INFO:version 2.3.0
2021-03-29 22:33:48,937:INFO:Initializing setup()
2021-03-29 22:33:48,937:INFO:setup(target=predict, ml_usecase=classification, available_plots={'parameter': 'Hyperparameters', 'auc': 'AUC', 'confusion_matrix': 'Confusion Matrix', 'threshold': 'Threshold', 'pr': 'Precision Recall', 'error': 'Prediction Error', 'class_report': 'Class Report', 'rfe': 'Feature Selection', 'learning': 'Learning Curve', 'manifold': 'Manifold Learning', 'calibration': 'Calibration Curve', 'vc': 'Validation Curve', 'dimension': 'Dimensions', 'feature': 'Feature Importance', 'feature_all': 'Feature Importance (All)', 'boundary': 'Decision Boundary', 'lift': 'Lift Chart', 'gain': 'Gain Chart', 'tree': 'Decision Tree'}, train_size=0.7, test_data=None, preprocess=True, imputation_type=simple, iterative_imputation_iters=5, categorical_features=None, categorical_imputation=constant, categorical_iterative_imputer=lightgbm, ordinal_features=None, high_cardinality_features=None, high_cardinality_method=frequency, numeric_features=None, numeric_imputation=mean, numeric_iterative_imputer=lightgbm, date_features=None, ignore_features=None, normalize=False, normalize_method=zscore, transformation=False, transformation_method=yeo-johnson, handle_unknown_categorical=True, unknown_categorical_method=least_frequent, pca=False, pca_method=linear, pca_components=None, ignore_low_variance=False, combine_rare_levels=False, rare_level_threshold=0.1, bin_numeric_features=None, remove_outliers=False, outliers_threshold=0.05, remove_multicollinearity=False, multicollinearity_threshold=0.9, remove_perfect_collinearity=True, create_clusters=False, cluster_iter=20, polynomial_features=False, polynomial_degree=2, trigonometry_features=False, polynomial_threshold=0.1, group_features=None, group_names=None, feature_selection=False, feature_selection_threshold=0.8, feature_selection_method=classic, feature_interaction=False, feature_ratio=False, interaction_threshold=0.01, fix_imbalance=False, fix_imbalance_method=None, transform_target=False, transform_target_method=box-cox, data_split_shuffle=True, data_split_stratify=False, fold_strategy=stratifiedkfold, fold=10, fold_shuffle=False, fold_groups=None, n_jobs=-1, use_gpu=False, custom_pipeline=None, html=True, session_id=123, log_experiment=False, experiment_name=None, log_plots=False, log_profile=False, log_data=False, silent=True, verbose=True, profile=False, profile_kwargs=None, display=None)
2021-03-29 22:33:48,937:INFO:Checking environment
2021-03-29 22:33:48,937:INFO:python_version: 3.8.0
2021-03-29 22:33:48,937:INFO:python_build: ('default', 'Nov 14 2019 22:29:45')
2021-03-29 22:33:48,937:INFO:machine: x86_64
2021-03-29 22:33:48,938:INFO:platform: Linux-5.4.0-1029-aws-x86_64-with-glibc2.2.5
2021-03-29 22:33:48,938:WARNING:cannot find psutil installation. memory not traceable. Install psutil using pip to enable memory logging.
2021-03-29 22:33:48,938:INFO:Checking libraries
2021-03-29 22:33:48,938:INFO:pd==0.25.2
2021-03-29 22:33:48,938:INFO:numpy==1.19.5
2021-03-29 22:33:48,938:INFO:sklearn==0.23.2
2021-03-29 22:33:48,938:INFO:lightgbm==3.2.0
2021-03-29 22:33:49,208:INFO:catboost==0.25
2021-03-29 22:33:49,218:INFO:xgboost==0.90
2021-03-29 22:33:49,797:INFO:mlflow==1.15.0
2021-03-29 22:33:49,798:INFO:Checking Exceptions
2021-03-29 22:33:49,798:INFO:Declaring global variables
2021-03-29 22:33:49,798:INFO:USI: 4af5
2021-03-29 22:33:49,798:INFO:pycaret_globals: {'X', 'USI', '_ml_usecase', 'y', 'X_test', 'transform_target_param', 'master_model_container', 'fold_shuffle_param', 'imputation_regressor', '_available_plots', 'n_jobs_param', '_gpu_n_jobs_param', 'y_test', 'exp_name_log', '_all_metrics', '_all_models_internal', 'create_model_container', 'gpu_param', 'transform_target_method_param', 'seed', '_all_models', 'fix_imbalance_method_param', 'fold_generator', 'html_param', 'display_container', 'imputation_classifier', 'target_param', 'fold_param', 'log_plots_param', 'fold_groups_param', '_internal_pipeline', 'experiment__', 'logging_param', 'X_train', 'stratify_param', 'pycaret_globals', 'prep_pipe', 'data_before_preprocess', 'y_train', 'iterative_imputation_iters_param', 'fix_imbalance_param'}
2021-03-29 22:33:49,798:INFO:Preparing display monitor
2021-03-29 22:33:49,798:INFO:Preparing display monitor
2021-03-29 22:33:49,804:INFO:Importing libraries
2021-03-29 22:33:49,805:INFO:Copying data for preprocessing
2021-03-29 22:33:49,807:INFO:Declaring preprocessing parameters
2021-03-29 22:33:49,809:INFO:Creating preprocessing pipeline
deleted-user-7226569 | 11 posts | March 29, 2021, 10:50 p.m. | permalink

Where do you execute that code?

Staff fjl | 4614 posts | PythonAnywhere staff | March 30, 2021, 8:20 a.m. | permalink

I execute Python 3.8

deleted-user-7226569 | 11 posts | March 30, 2021, 11:32 a.m. | permalink

Is it a task? webapp? jupyter notebook? a script in the console?

Staff fjl | 4614 posts | PythonAnywhere staff | March 30, 2021, 3:04 p.m. | permalink

It is a file.py when I click the Run button show me this mensages. This code is working in my local computer..

I think that "pip install pycaret" is not installed corretly.

The simple code bellow not run too:

from pycaret.datasets import get_data
from pycaret.classification import *
juice = get_data('juice') 
exp_clf = setup(juice, target = 'Purchase', silent=True)
compare_models()

Please, I need someone to help me execute the code and show me step by step about pycaret installation.

Notes:

My english is work in progress :)

I will create a task in the future

deleted-user-7226569 | 11 posts | March 30, 2021, 5:10 p.m. | permalink

When you run a program from the editor, it will run with a specific Python version based on your account -- for you, that is Python 3.8.

However, because the operating system requires it, pip installs packages for Python 2.7. (We're working on a new system image that should avoid that confusing setup.)

So, in order to install a package and have it work when you run the program from the editor, you should use the pip3.8 command to install it -- for example:

pip3.8 install --user pycaret

See this help page for more information.

Staff giles | 12096 posts | PythonAnywhere staff | March 30, 2021, 5:25 p.m. | permalink

Hi everybody,

I installed pycaret with success.

But .. I have this problem now:

exception calling callback for <Future at 0x7fcb49146880 state=finished raised BrokenProcessPool>
joblib.externals.loky.process_executor._RemoteTraceback: 
'''
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 391, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/usr/lib/python3.8/site-packages/ipyparallel/serialize/codeutil.py", line 24, in code_ctor
    return types.CodeType(*args)
TypeError: an integer is required (got type bytes)
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/_base.py", line 625, in _invoke_callbacks
    callback(self)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 340, in __call__
    self.parallel.dispatch_next()
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 768, in dispatch_next
    if not self.dispatch_one_batch(self._original_iterator):
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 834, in dispatch_one_batch
    self._dispatch(tasks)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 753, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/usr/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 543, in apply_async
    future = self._workers.submit(SafeFunction(func))
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 159, in submit
    return super(_ReusablePoolExecutor, self).submit(
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1027, in submit
    raise self._flags.broken
joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
exception calling callback for <Future at 0x7fcb4910b310 state=finished raised BrokenProcessPool>
joblib.externals.loky.process_executor._RemoteTraceback: 
'''
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 391, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/usr/lib/python3.8/site-packages/ipyparallel/serialize/codeutil.py", line 24, in code_ctor
    return types.CodeType(*args)
TypeError: an integer is required (got type bytes)
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/_base.py", line 625, in _invoke_callbacks
    callback(self)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 340, in __call__
    self.parallel.dispatch_next()
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 768, in dispatch_next
    if not self.dispatch_one_batch(self._original_iterator):
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 834, in dispatch_one_batch
    self._dispatch(tasks)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 753, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/usr/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 543, in apply_async
    future = self._workers.submit(SafeFunction(func))
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 159, in submit
    return super(_ReusablePoolExecutor, self).submit(
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1027, in submit
    raise self._flags.broken
joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
exception calling callback for <Future at 0x7fcb490f4e20 state=finished raised BrokenProcessPool>
joblib.externals.loky.process_executor._RemoteTraceback: 
'''
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 391, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/usr/lib/python3.8/site-packages/ipyparallel/serialize/codeutil.py", line 24, in code_ctor
    return types.CodeType(*args)
TypeError: an integer is required (got type bytes)
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/_base.py", line 625, in _invoke_callbacks
    callback(self)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 340, in __call__
    self.parallel.dispatch_next()
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 768, in dispatch_next
    if not self.dispatch_one_batch(self._original_iterator):
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 834, in dispatch_one_batch
    self._dispatch(tasks)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 753, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/usr/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 543, in apply_async
    future = self._workers.submit(SafeFunction(func))
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 159, in submit
    return super(_ReusablePoolExecutor, self).submit(
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1027, in submit
    raise self._flags.broken
joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
exception calling callback for <Future at 0x7fcb490f4850 state=finished raised BrokenProcessPool>
joblib.externals.loky.process_executor._RemoteTraceback: 
'''
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 391, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/usr/lib/python3.8/site-packages/ipyparallel/serialize/codeutil.py", line 24, in code_ctor
    return types.CodeType(*args)
TypeError: an integer is required (got type bytes)
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/_base.py", line 625, in _invoke_callbacks
    callback(self)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 340, in __call__
    self.parallel.dispatch_next()
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 768, in dispatch_next
    if not self.dispatch_one_batch(self._original_iterator):
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 834, in dispatch_one_batch
    self._dispatch(tasks)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 753, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/usr/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 543, in apply_async
    future = self._workers.submit(SafeFunction(func))
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 159, in submit
    return super(_ReusablePoolExecutor, self).submit(
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1027, in submit
    raise self._flags.broken
joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
exception calling callback for <Future at 0x7fcb4913d940 state=finished raised BrokenProcessPool>
joblib.externals.loky.process_executor._RemoteTraceback: 
'''
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 391, in _process_worker
    call_item = call_queue.get(block=True, timeout=timeout)
  File "/usr/lib/python3.8/multiprocessing/queues.py", line 116, in get
    return _ForkingPickler.loads(res)
  File "/usr/lib/python3.8/site-packages/ipyparallel/serialize/codeutil.py", line 24, in code_ctor
    return types.CodeType(*args)
TypeError: an integer is required (got type bytes)
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/_base.py", line 625, in _invoke_callbacks
    callback(self)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 340, in __call__
    self.parallel.dispatch_next()
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 768, in dispatch_next
    if not self.dispatch_one_batch(self._original_iterator):
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 834, in dispatch_one_batch
    self._dispatch(tasks)
  File "/usr/lib/python3.8/site-packages/joblib/parallel.py", line 753, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/usr/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 543, in apply_async
    future = self._workers.submit(SafeFunction(func))
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 159, in submit
    return super(_ReusablePoolExecutor, self).submit(
  File "/usr/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1027, in submit
    raise self._flags.broken
joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable.
deleted-user-7226569 | 11 posts | March 31, 2021, 8:46 p.m. | permalink

It's hard to say exactly what is going on there without knowing more about your code. The error is saying that something is putting something into a queue to be run by a subprocess, but that thing is not pickleable so it can't be serialized to be put onto the queue.

I see that it appears to be happening deep in the loky library; what code do you have that interacts with that library?

Staff giles | 12096 posts | PythonAnywhere staff | April 1, 2021, 9:57 a.m. | permalink

I'm trying to run the simple code that the pycaret library offers

from pycaret.datasets import get_data
from pycaret.classification import *
juice = get_data('juice') 
exp_clf = setup(juice, target = 'Purchase', silent=True)
compare_models()
deleted-user-7226569 | 11 posts | April 1, 2021, 11:42 a.m. | permalink

Hi, it looks like something related to the pycaret setup and not specifically to PythonAnywhere. I see pycaret has a web site with many examples and a slack channel where you can ask questions, maybe you should try asking there?

Staff pafk | 3226 posts | PythonAnywhere staff | April 1, 2021, 3:49 p.m. | permalink