Configuring a Server for Datameer Failover

Owned by Charles Bishop (Deactivated)
Jan 30, 2018
2 min read

Introduction

Failover is an operational mode in which the functions of a system component (i.e. the server or database) are taken over by secondary system components when the primary component is unavailable - either through failure or scheduled downtime. This involves offloading tasks to a standby system component making the procedure as seamless as possible for the end user. You must be aware of the following when configuring Datameer for failover.

Configuring a Server for High Availability

Use MySQL - High Availability and Scalability to find different possible solutions for configuring Datameer for high availability. 

Configuring a Server for Datameer Failover

To configure Datameer for failover, you need to prepare a backup server that has Datameer installed. This backup server can be started-up whenever the regular Datameer server shuts down. If both servers are configured to use the same MySQL database, the backup server can be started immediately after the primary server shuts down without any further configuration. You must ensure that only one Datameer server is running at a time. Never start both servers at the same time.

Prerequisites

  • You require an additional license file for the backup server
  • The library and configuration files for both servers must be synchronized (etc/das-env.sh, etc/custom-jars/*, etc/custom-plugins/* and conf/*)

Configuring a Server for MySQL Crashes

  • Backup the Datameer MySQL database regularly (e.g. every 15 minutes):

    mysqldump [-h <dbhost>] -u dap dap -p > <backup-folder>/das-backup.dmp

  • Delay deleting files from HDFS for a period longer than this backup interval, by setting this property in conf/live.properties:

    # To allow better failover in case of a crashed db deleted data should be kept longer than the configured
# frequency of database backups
housekeeping.keep-deleted-data=30m

How to recover after a MySQL crash:

  1. Stop Datameer

    bin/conductor.sh stop

  2. Ensure that the Datameer database is present on the MySQL failover machine or run the following command:

    mysql [-h <dbhost>] -u root -p < bin/mysql-init.sql

  3. Restore the database from the MySQL dump using the following command:

    mysql -u dap -p dap < <path-to>/das-backup.dmp
  4. Direct Datameer to the new database location (if it is different).

  5. Start Datameer.

    bin/conductor.sh startConfiguring a server for high availabity