Host configuration

Most Hadoop services come with web interfaces for administration and/or monitoring purposes. In TDP, all services are kerberized by default. Thus, specific configurations need to be applied to the host to access the web interfaces.

This page provides instructions to configure the host and access web interfaces.

Requirements

This tutorial assumes you are running a cluster based on TDP getting started, an easy-to-launch TDP environment for testing purposes. This deployment provides you with tdp_user, a user with the ability to kinit for authentication.

Note: When using another TDP deployment than tdp-getting-started, some commands require adaptation to your deployment.

/etc/hosts configuration

The virtual machines are addressable by their hostname and FQDNs inside the cluster because TDP updates their /etc/hosts file. However, the host machine doesn’t know about it. There are three main benefits to configuring the host machine /etc/hosts file:

  • To access the web UIs
  • For krb5.conf configuration file, see next section
  • To use the ssh command directly, instead of the vagrant ssh command, and to work with commands such as sftp, rsync, …

Hosts configuration for the TDP getting started can be found in ./ tdp-vagrant/vagrant.yml from the tdp-getting-started local folder. The ./scripts/helper.sh script can generate the list of hosts in the cluster. Add the generated lines to your /etc/hosts file to resolve the local nodes from your shell or browser.

# Pipe hosts definition from script output to hosts file
./scripts/helper.sh -h | sudo tee -a /etc/hosts

Kerberos configuration

tdp_user needs to be authenticated on the host to access the services. To do so:

  1. Install Kerberos KDF client (krb5-user) on the host.

    sudo apt update && sudo apt install krb5-user

    When prompted for a realm name, enter REALM.TDP. This value can be modified later.

  2. Edit the /etc/krb5.conf file to contain the configuration used in TDP.

    [libdefaults]
default_ccache_name = /tmp/krb5cc_%{uid}

[realms]
REALM.TDP = {
kdc = master-01.tdp
admin_server = master-01.tdp
}

  3. Authenticate tdp_user with his Kerberos principal.

    Use the kinit command from the tdp-getting-started local folder to obtain or renew a Kerberos ticket. Using the password (tdp_user123):

    echo 'tdp_user123' | kinit tdp_user@REALM.TDP

Web Browser configuration

Configure SPNEGO to make Kerberos work over HTTP for the web browser.

For Firefox:

  1. Navigate to the about:config page
  2. In the search box, look for network.negotiate-auth.trusted-uris, click on Edit
  3. Enter the hostname or the domain of the web server that is protected by Kerberos HTTP SPNEGO. Separate multiple domains and hostnames with a comma. In our case, simply add .tdp top-level domain (TLD).
  4. Click Save

Add required certificates to Web Browser

Finally, adding PEM certificates to the web browser allows to avoid security issue pages:

For Firefox:

  1. Navigate to Settings
  2. In the search bar look for Certificates. Select the first option: View Certificates
  3. Switch to the Authorities tab and import the tdp_ca.crt certificate located on ./ansible_collections/tosit/tdp_prerequisites/roles/certificates/files/tdp_getting_started_certs of the tdp-getting-started local folder
  4. Check the box Trust this CA to identify websites
  5. Restart the browser

Web interfaces are accessible at those addresses: