Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Enable Spark Plugin

I am trying to enable Spark plugin , with not much luck. Updating the “plugins” variable in sahara.conf doesn’t seem to have any effect. Running "tox -e venv -- sahara-api --config-file etc/sahara/sahara.conf --debug " and adding “spark” to default array, in plugins/base.py, returns with the following error:

    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'vanilla' loaded (sahara.plugins.vanilla.plugin:VanillaProvider)
    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'idh' loaded (sahara.plugins.intel.plugin:IDHProvider)
    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'hdp' loaded (sahara.plugins.hdp.ambariplugin:AmbariPlugin)
    2014-08-11 21:48:37.285 19329 CRITICAL sahara [-] RuntimeError: Plugins couldn't be loaded: spark
    2014-08-11 21:48:37.285 19329 TRACE sahara Traceback (most recent call last):
    2014-08-11 21:48:37.285 19329 TRACE sahara   File ".tox/venv/bin/sahara-api", line 14, in <module>
    2014-08-11 21:48:37.285 19329 TRACE sahara     sys.exit(main())
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/cli/sahara_api.py", line 60, in main
    2014-08-11 21:48:37.285 19329 TRACE sahara     app = server.make_app()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/main.py", line 107, in make_app
    2014-08-11 21:48:37.285 19329 TRACE sahara     plugins_base.setup_plugins()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 139, in setup_plugins
    2014-08-11 21:48:37.285 19329 TRACE sahara     PLUGINS = PluginManager()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 87, in __init__
    2014-08-11 21:48:37.285 19329 TRACE sahara     self._load_cluster_plugins()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 112, in _load_cluster_plugins
    2014-08-11 21:48:37.285 19329 TRACE sahara     % ", ".join(requested_plugins - loaded_plugins))
    2014-08-11 21:48:37.285 19329 TRACE sahara RuntimeError: Plugins couldn't be loaded: spark
    2014-08-11 21:48:37.285 19329 TRACE saharaERROR: InvocationError: '/home/navesta/OpenStack/sahara/.tox/venv/bin/sahara-api --config-file etc/sahara/sahara.conf --debug'
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   venv: commands failed

Running "tox -e venv -- sahara-all --config-file etc/sahara/sahara.conf --debug", returns the following error messages:

venv develop-inst-nodeps: /home/navesta/OpenStack/sahara
venv runtests: commands[0] | sahara-all --config-file etc/sahara/sahara.conf --debug
Traceback (most recent call last):
  File ".tox/venv/bin/sahara-all", line 6, in <module>
    from sahara.cli.sahara_all import main
  File "/home/navesta/OpenStack/sahara/sahara/cli/sahara_all.py", line 19, in <module>
AttributeError: 'module' object has no attribute 'patch_all'
ERROR: InvocationError: '/home/navesta/OpenStack/sahara/.tox/venv/bin/sahara-all --config-file etc/sahara/sahara.conf --debug'
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   venv: commands failed

I am on icehouse branch of git:

navesta@UbuntuMaas:~/OpenStack/sahara$ tox -e venv -- sahara-api --version
venv develop-inst-nodeps: /home/navesta/OpenStack/sahara
venv runtests: commands[0] | sahara-api --version
2014.1.2
_____________________________________________________________________________________ summary ______________________________________________________________________________________
  venv: commands succeeded
  congratulations :)

Any help would be appreciated.

Enable Spark Plugin

I am trying to enable Spark plugin , with not much luck. Updating the “plugins” variable in sahara.conf doesn’t seem to have any effect. Running "tox -e venv -- sahara-api --config-file etc/sahara/sahara.conf --debug " and adding “spark” to default array, in plugins/base.py, returns with the following error:

    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'vanilla' loaded (sahara.plugins.vanilla.plugin:VanillaProvider)
    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'idh' loaded (sahara.plugins.intel.plugin:IDHProvider)
    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'hdp' loaded (sahara.plugins.hdp.ambariplugin:AmbariPlugin)
    2014-08-11 21:48:37.285 19329 CRITICAL sahara [-] RuntimeError: Plugins couldn't be loaded: spark
    2014-08-11 21:48:37.285 19329 TRACE sahara Traceback (most recent call last):
    2014-08-11 21:48:37.285 19329 TRACE sahara   File ".tox/venv/bin/sahara-api", line 14, in <module>
    2014-08-11 21:48:37.285 19329 TRACE sahara     sys.exit(main())
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/cli/sahara_api.py", line 60, in main
    2014-08-11 21:48:37.285 19329 TRACE sahara     app = server.make_app()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/main.py", line 107, in make_app
    2014-08-11 21:48:37.285 19329 TRACE sahara     plugins_base.setup_plugins()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 139, in setup_plugins
    2014-08-11 21:48:37.285 19329 TRACE sahara     PLUGINS = PluginManager()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 87, in __init__
    2014-08-11 21:48:37.285 19329 TRACE sahara     self._load_cluster_plugins()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 112, in _load_cluster_plugins
    2014-08-11 21:48:37.285 19329 TRACE sahara     % ", ".join(requested_plugins - loaded_plugins))
    2014-08-11 21:48:37.285 19329 TRACE sahara RuntimeError: Plugins couldn't be loaded: spark
    2014-08-11 21:48:37.285 19329 TRACE saharaERROR: InvocationError: '/home/navesta/OpenStack/sahara/.tox/venv/bin/sahara-api --config-file etc/sahara/sahara.conf --debug'
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   venv: commands failed

Running "tox -e venv -- sahara-all --config-file etc/sahara/sahara.conf --debug", returns the following error messages:

venv develop-inst-nodeps: /home/navesta/OpenStack/sahara
venv runtests: commands[0] | sahara-all --config-file etc/sahara/sahara.conf --debug
Traceback (most recent call last):
  File ".tox/venv/bin/sahara-all", line 6, in <module>
    from sahara.cli.sahara_all import main
  File "/home/navesta/OpenStack/sahara/sahara/cli/sahara_all.py", line 19, in <module>
AttributeError: 'module' object has no attribute 'patch_all'
ERROR: InvocationError: '/home/navesta/OpenStack/sahara/.tox/venv/bin/sahara-all --config-file etc/sahara/sahara.conf --debug'
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   venv: commands failed

I am on icehouse branch of git:

navesta@UbuntuMaas:~/OpenStack/sahara$ tox -e venv -- sahara-api --version
venv develop-inst-nodeps: /home/navesta/OpenStack/sahara
venv runtests: commands[0] | sahara-api --version
2014.1.2
_____________________________________________________________________________________ summary ______________________________________________________________________________________
  venv: commands succeeded
  congratulations :)

Any help would be appreciated.

Edit 1: As per dmitrymex comment, Spark is not avaiable on the main branch. did the following to get it to show up in Horizon:

1- Checkout main branch
2- source .tox/venv/bin/activate
3- pip install oslo.db
4- pip install -r requirements.txt
5- tox -e venv -- sahara-db-manage --config-file etc/sahara/sahara.conf upgrade head
6- tox -e venv -- sahara-all --config-file etc/sahara/sahara.conf --debug

Admittedly, haven’t done any further testing. Will post if any anything interesting shows up.

Enable Spark Plugin

I am trying to enable Spark plugin , with not much luck. Updating the “plugins” variable in sahara.conf doesn’t seem to have any effect. Running "tox -e venv -- sahara-api --config-file etc/sahara/sahara.conf --debug " and adding “spark” to default array, in plugins/base.py, returns with the following error:

    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'vanilla' loaded (sahara.plugins.vanilla.plugin:VanillaProvider)
    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'idh' loaded (sahara.plugins.intel.plugin:IDHProvider)
    2014-08-11 21:48:37.285 19329 INFO sahara.plugins.base [-] Plugin 'hdp' loaded (sahara.plugins.hdp.ambariplugin:AmbariPlugin)
    2014-08-11 21:48:37.285 19329 CRITICAL sahara [-] RuntimeError: Plugins couldn't be loaded: spark
    2014-08-11 21:48:37.285 19329 TRACE sahara Traceback (most recent call last):
    2014-08-11 21:48:37.285 19329 TRACE sahara   File ".tox/venv/bin/sahara-api", line 14, in <module>
    2014-08-11 21:48:37.285 19329 TRACE sahara     sys.exit(main())
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/cli/sahara_api.py", line 60, in main
    2014-08-11 21:48:37.285 19329 TRACE sahara     app = server.make_app()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/main.py", line 107, in make_app
    2014-08-11 21:48:37.285 19329 TRACE sahara     plugins_base.setup_plugins()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 139, in setup_plugins
    2014-08-11 21:48:37.285 19329 TRACE sahara     PLUGINS = PluginManager()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 87, in __init__
    2014-08-11 21:48:37.285 19329 TRACE sahara     self._load_cluster_plugins()
    2014-08-11 21:48:37.285 19329 TRACE sahara   File "/home/navesta/OpenStack/sahara/sahara/plugins/base.py", line 112, in _load_cluster_plugins
    2014-08-11 21:48:37.285 19329 TRACE sahara     % ", ".join(requested_plugins - loaded_plugins))
    2014-08-11 21:48:37.285 19329 TRACE sahara RuntimeError: Plugins couldn't be loaded: spark
    2014-08-11 21:48:37.285 19329 TRACE saharaERROR: InvocationError: '/home/navesta/OpenStack/sahara/.tox/venv/bin/sahara-api --config-file etc/sahara/sahara.conf --debug'
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   venv: commands failed

Running "tox -e venv -- sahara-all --config-file etc/sahara/sahara.conf --debug", returns the following error messages:

venv develop-inst-nodeps: /home/navesta/OpenStack/sahara
venv runtests: commands[0] | sahara-all --config-file etc/sahara/sahara.conf --debug
Traceback (most recent call last):
  File ".tox/venv/bin/sahara-all", line 6, in <module>
    from sahara.cli.sahara_all import main
  File "/home/navesta/OpenStack/sahara/sahara/cli/sahara_all.py", line 19, in <module>
AttributeError: 'module' object has no attribute 'patch_all'
ERROR: InvocationError: '/home/navesta/OpenStack/sahara/.tox/venv/bin/sahara-all --config-file etc/sahara/sahara.conf --debug'
_____________________________________________________________________________________ summary ______________________________________________________________________________________
ERROR:   venv: commands failed

I am on icehouse branch of git:

navesta@UbuntuMaas:~/OpenStack/sahara$ tox -e venv -- sahara-api --version
venv develop-inst-nodeps: /home/navesta/OpenStack/sahara
venv runtests: commands[0] | sahara-api --version
2014.1.2
_____________________________________________________________________________________ summary ______________________________________________________________________________________
  venv: commands succeeded
  congratulations :)

Any help would be appreciated.

Edit 1: As per dmitrymex comment, Spark is not avaiable on the main icehouse branch. did the following to get it to show up in Horizon:

1- Checkout main branch
2- source .tox/venv/bin/activate
3- pip install oslo.db
4- pip install -r requirements.txt
5- tox -e venv -- sahara-db-manage --config-file etc/sahara/sahara.conf upgrade head
6- tox -e venv -- sahara-all --config-file etc/sahara/sahara.conf --debug

Admittedly, haven’t done any further testing. Will post if any anything interesting shows up.