Skip to content
Fiction Becomes Fact

Hop server container setup

Introduction

Apache Hop is an open source data integration platform and a fork of Pentaho Data Integration. In recent years I have enjoyed using Pentaho both at home and professionally and I’m very excited about Hop. For those who know Pentaho, much will be familiar. You can read about the differences (and similarities) here.

Setup

I will use a long-lived Apache Hop server container to experiment with. With this container setup it is possible to run pipelines and workflows remotely, but also to run web services using a project and environments. Thanks to the Hop team by the way.

  1. Create the folders needed by the container

    Run the following commands inside your home folder:

    # Open your terminal application
    cd ~
    mkdir -p hop/data/env
    mkdir -p hop/data/metadata
    mkdir -p hop/data/jdbc
    mkdir -p hop/data/log
    mkdir -p hop/data/projects
    cd hop
  2. Create the script needed to run the container

    Save the following script as hop_run.sh:

    hop_run.sh
    # To create this script use your text editor application, for example Nano
    docker run -d \
    --name=hop \
    --hostname=hop \
    -p 8182:8182 \
    -v $PWD/data:/files \
    -e HOP_SERVER_USER="<username>" \
    -e HOP_SERVER_PASS="<password>" \
    -e HOP_SERVER_PORT=8182 \
    -e HOP_SERVER_HOSTNAME=0.0.0.0 \
    -e HOP_PROJECT_NAME="<project_name>" \
    -e HOP_PROJECT_FOLDER="/files/projects/<project_folder_name>" \
    -e HOP_ENVIRONMENT_NAME="<env_name>" \
    -e HOP_ENVIRONMENT_CONFIG_FILE_NAME_PATHS="/files/env/<env_path>.json" \
    -e HOP_SERVER_METADATA_FOLDER="<metadata_location>" \
    -e HOP_SHARED_JDBC_FOLDER="/files/jdbc" \
    -e HOP_LOG_PATH="/files/log/hop.err.log" \
    -e TZ=Europe/Amsterdam \
    --restart unless-stopped \
    apache/hop
    # IMPORTANT: Please read the instructions below
    Instructions:
    • Optional Replace docker with podman if needed
    • Optional Replace $PWD/data with the location of the Hop server files. This can be a fileserver mount
    • Optional Replace <username> and <password> with your username and password if needed
    • Required Replace port number 8182 everywhere with a port number that is available if needed.
    • Required Replace <project_name> with the name of your project. If you only use Hop server to run pipelines or workflows remotely, then remove this environment variable
    • Required Replace <project_folder_name> with the name of your project folder. If you only use Hop server to run pipelines or workflows remotely, then remove this environment variable
    • Required Replace <env_name> with the name of your Hop environment. For example development-config. If you only use Hop server to run pipelines or workflows remotely, then remove this environment variable
    • Required Replace <env_path> with your Hop environment path. For example /files/env/hop-server-test-development-config.json. If you only use Hop server to run pipelines or workflows remotely, then remove this environment variable
    • Required Replace <metadata_location> with the location of the metadata. If you only use Hop Server to run pipelines or workflows remotely this can be /files/metadata, but when using a project this could be /files/projects/<project_folder_name>/metadata
    • Optional Replace /files/jdbc with the location of the JDBC drivers that are not included by default
    • Optional Replace /files/log/hop.err.log with the path to your error log. While this environment variable is optional, I prefer to have it so I can easily access the log files from a shared folder
    • Required Replace Europe/Amsterdam with your own timezone
  3. Run the script to create the container

    Run the following command:

    # Open your terminal application
    sudo sh hop_run.sh

    The image apache/hop is automatically pulled and the container is created.

  4. Check the results

    If needed you can check if the container is running properly.

    Now you can log in with your username and password and access the Hop server status by opening a web browser and going to: http://localhost:8182/hop/status. Replace localhost with the relevant IP address or FQDN if needed, and adjust the port if you changed it earlier.

    This will give you an overview of the pipelines and workflows after these are executed through the server.

Favorites

Hop Documentation

Comments

    No comments found for this note.

    Join the discussion for this note on Github. Comments appear on this page instantly.

    Tags:
    Copyright 2021- Fiction Becomes Fact