How to get Spark running Standalone?

Hello follow Linode users. For the past week or so, I've been trying and failing to configure my Spark Java webserver to work standalone on my Linode. Since I am only operating one Linode, the tutorial for hadoop and spark has been unhelpful for me. When I launch it, the server starts, then in a split-second, stops. Maybe it's my startscript that's flawed? Please see below:

export SPARK_MASTER_IP='mylinodeip'
export SPARK_LOCAL_IP='127.0.0.0'
export SPARK_MASTER_PORT=80
export SPARK_WORKER_MEMORY='100MB'
export STANDALONE_SPARK_MASTER_HOST='mylinodeip'
java -jar server.jar

Am I doing anything wrong? If anyone has any insights, I would greatly appreciate them.

Thanks in advance!

EDIT: The webserver seems to stay alive ONLY when I launch it from the script, but I cannot connect to it on the port I specified. If I launch the jar itself without any of the above parameters (which would mean it runs on port 80 at 127.0.0.0), it just stops as soon as it starts.

0 Replies

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct