Big Data 256 - Atlas Installation

Official Installation

Extract

cd /opt/software/apache-atlas-sources-1.2.0/distro/target

# Extract
tar zxvf apache-atlas-1.2.0-bin.tar.gz

mv apache-atlas-1.2.0/ /opt/servers/atlas-1.2.0

Environment Variables

Modify /etc/profile to set the ATLAS_HOME environment variable:

vim /etc/profile

# atlas
export ATLAS_HOME=/opt/servers/atlas-1.2.0
export PATH=$PATH:ATLAS_HOME/bin

Start Service

After refreshing environment variables, start the service:

cd $ATLAS_HOME/bin
./atlas_start.py

Check Process

ps -ef | grep atlas

Stop Service

cd $ATLAS_HOME/bin
./atlas_stop.py

Check Solr

cd /opt/servers/atlas-1.2.0/solr/bin
./solr status

Solr process 25038 running on port 9838

Access Web UI

http://h122.wzk.icu:21000

Login username and password: admin

Hive Lineage Import

Copy Configuration

Copy $ATLAS_HOME/conf/atlas-application.properties to $HIVE_HOME/conf directory:

ln -s $ATLAS_HOME/conf/atlas-application.properties $HIVE_HOME/conf/atlas-application.properties

Copy Jar Packages

ln -s $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar $ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-base-2.9.9.jar

ln -s $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar $ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-json-provider-2.9.9.jar

ln -s $ATLAS_HOME/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar $ATLAS_HOME/hook/hive/atlas-hive-plugin-impl/jackson-module-jaxb-annotations-2.9.9.jar

Modify Hive

vim $HIVE_HOME/conf/hive-site.xml

Add hook:

<property>
  <name>hive.exec.post.hooks</name>
  <value>org.apache.atlas.hive.hook.HiveHook</value>
</property>

Add HIVE_AUX_JARS_PATH variable in $HIVE_HOME/conf/hive-env.sh:

# If not created before
mv $HIVE_HOME/conf/hive-env.sh.template $HIVE_HOME/conf/hive-env.sh
vim $HIVE_HOME/conf/hive-env.sh

Write the following content:

export HIVE_AUX_JARS_PATH=/opt/servers/atlas-1.2.0/hook/hive

Batch Import

We need Hive to start normally. During execution, you will need to input the admin/admin username and password:

# Execute hive in console
sh /opt/servers/atlas-1.2.0/bin/import-hive.sh

Access Results

http://h122.wzk.icu:21000

Hive Hook can capture the following operations:

  • create database
  • create table/view, create table as select
  • load, import, export
  • DMLs(insert)
  • alter database
  • alter table
  • alter view