Ranger admin dev overview

Apache Ranger
Comprehensive Security for
Enterprise Hadoop
Developer Overview
Presenter:
Tushar Dudhatra

Agenda
➔Security Approach within Hadoop
➔Core Features Of Ranger
➔Technology Stack
➔Ranger Architecture
◆ High Level Overview
◆ Code flow
➔How to build and run ?

Security
Approach
within Hadoop
From Slide No: #4
To Slide No: #6

Security today in Hadoop with HDP
HDP2.3
Centralized Security Administration with Ranger
Authentication
Who am I ?
prove it !
Authorization
What can I do?
Audit
What did I do?
Data Protection
Can data be encrypted at
rest and over the wire?
Fine grain access
control with Apache
Ranger
Centralized audit
reporting with
Apache Ranger
➔ Wire
encryption in
Hadoop
➔ Native and
partner
encryption
➔ Kerberos
➔ API security
with Apache
Knox

Example Access Scenario With Ranger
Plugins Download
Defined Policies from
Centralized Policy store
Plugins Write to
Centralized
Audit store

Core Features
Of Ranger
From Slide No: #8
To Slide No: #8

Core Features Of Ranger
>> Authorization Support for HDFS, Hive, HBase, Knox, Storm, Solr, Kafka, Yarn
>> Audit destination support for HDFS, DB, SOLR (Including SolrCloud too)
>> Support for High Availability (HA)
>> Support for Transparent Data Encryption with KMS implementation
>> Dynamic policies with many other attributes
>> Stack based implementation of Plugins
>> Internal permission model within Ranger Admin UI
>> Tighter integration with Apache Ambari
>> Policy Manager supports six different databases

Technology Stack
From Slide No: #10
To Slide No: #11

Ranger Implementation - components
➔ Central Interface for security
administration.
➔ Users can manage policies, view
audit activities, manage users and
permissions
➔ Sync utility to pull users and groups
from Unix or LDAP or AD.
➔ User/group info is stored within
Ranger Admin DB and used for
policy definitions.
➔ Lightweight Java programs within Hadoop components.
➔ Pull in policies from policy admin and store them locally in cache.
➔ Act as Authorization module and evaluate user requests against security policies before
granting access.
➔ Collect data from user request and store this data into the audit store.
Ranger Policy-Admin Server Ranger User-Sync Server
Ranger Security Plugins

Ranger Policy-Admin Tool
Backbone.Model REST Interface
Client Server
DB➔ Backbone
➔ jQuery
➔ Underscore js
➔ Marionette
➔ Bootstrap
➔ Spring Framework
➔ Spring Security
➔ EclipseLink
➔ JPA
➔ tomcat
➔ Apache Commons
➔ Jersey
➔ JUnit
➔ DB Connector
➔ log4j
Database could be:
➔ MySql
➔ Oracle
➔ MS-SQL Server
➔ SQLAnywhere
➔ Postgres

Ranger Architecture
From Slide No: #12
To Slide No: #17

Typical Flow of Ranger Admin
UI Server DB
Backbone.js
html
css
REST Mgr Service DAO
MySql
Postgres
Oracle
SQLA
MSSQL

3 Pillars of Ranger Admin (Policy Manager)
Service Def
Service-1 Service-3Service-2
Policy-1 Policy-2 Policy-3
Policy-4 Policy-5 Policy-6

Ranger HDFS Service Def
{
"id":1,
"name": "hdfs",
"implClass":"RangerServiceHdfs",
"label": "HDFS Repository",
"description": "HDFS Repository",
"guid": "8e9b-d5d377284b2d",
"resources": [
{
"itemId": 1,
"name": "path",
"type": "path",
"level": 10,
"parent": "",
"mandatory": true,
"lookupSupported": true,
"recursiveSupported": true,
"excludesSupported": false,
"matcher":"RangerPathReurceMatcher",
"matcherOptions": {
"wildCard":true, "ignoreCase":false },
"validationRegEx":"",
"validationMessage": "",
"uiHint":"",
"label": "Resource Path",
"description": "HDFS file path"
}],
"accessTypes": [
{
"itemId": 1,
"name":
"read",
"label": "Read"
},
{
"itemId": 2,
"name":
"write",
"label":
"Write"
},
{
"itemId": 3,
"name":
"execute",
"label":
"Execute"
}],
"configs": [
{
"itemId": 1,
"name":
"username",
"type":
"string",
"subType": "",
"mandatory":
"enums": [
{
"itemId": 1,
"name":
"authnType",
"elements":
[{
"itemId": 1,
"name": "simple",
"label": "Simple"
},
{
"itemId": 2,
"name": "kerberos",
"label": "Kerberos"
}],
"defaultIndex": 0
}],
"contextEnrichers": [],
"policyConditions": []
}

Example Policy Creation
{
"service": "hadoopdev",
"name": "policy-1",
"isEnabled": true,
"description": "securing /apps directory",
"isAuditEnabled": true,
"resources": {
"path": {
"values": ["/apps"],
"isRecursive": true
}
},
"policyItems": [
{
"users": ["admin"],
"delegateAdmin": true,
"accesses": [
{
"type": "read",
"isAllowed": true
},
{
"type": "write",
"isAllowed": true
},
{
"type": "execute",
"isAllowed": true
}]}]
}
Continue...

Example Policy Creation
XXPolicyDao
B
A
S
E
D
A
O
XXPolicyItemDao
XXPolicyResDao
XXPolicyResMap
Dao
ServiceREST ServiceDBStore RangerPolicyService
RangerPolicy → XXPolicy
D
A
T
A
B
A
S
E
Create RangerPolicy
Now Create Child Objects
i.e
Create XXPolicyItem
Create XXPolicyRes
Create XXPolicyResMap

How to Build and Run?
From Slide No: #19
To Slide No: #22

Prerequisites:
➔ JDK 7+
➔ Database you need anyone of these for Policy/Audit DB:
◆ MySQL (5.6+)
◆ ORACLE DB (11g+)
◆ Postgres (8.4+)
◆ MSSQL Server (2012)
◆ SQLAnywhere (17+)
➔ Maven (3.2 +)
➔ Ranger Admin process requires approximately 1.5GB of RAM

Building Ranger From Source
➔ Get the ranger source
◆ git clone -b master git@github.com:apache/incubator-ranger.git ranger
➔ Now build the source
◆ mvn clean compile package assembly:assembly -e
➔ Verify all the tar files under target dir i.e
◆ ranger-{version}-{component}-plugin.tar.gz
◆ ranger-{version}-admin.tar.gz
◆ ranger-{version}-usersync.tar.gz

Install/Configure Ranger Admin
➔ Lay down the binaries into appropriate places.
➔ Untar ranger-{version}-admin.tar.gz
➔ Open install.properties and update it to set values DB name, DB host, DB username:password and
other required details
➔ Once all the required properties are updated, execute setup.sh to run setup
➔ Start the Ranger Admin
➔ service ranger-admin start
➔ You can verify by visiting the external URL of the server using browser, for example :

Install And Configure Other Components of
Ranger
➔ After installing ranger-admin, next step is to configure and install ranger-usersync.
➔ After installing ranger-usersync, depending upon your need you can enable required ranger-
plugins.
➔ To configure and install usersync, you need to follow the same steps; i.e Update install.properties
as per your setup and then run setup.sh and start ranger-usersync service.
➔ To configure plugins, you need update install.properties and run enable-{complonent}-plugin.sh
and restart respective component to make ranger-plugin effective.
➔ For detailed documentation guide of installation, you can visit:
https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide

Typical things to
watch out during
Development
From Slide No: #23
To Slide No: #23

Some Healthy tips for Ranger Developers
➔ When you create a new file, don’t forget to add Apache Licence on top of the file. Licence is
required for all the files i.e .java, .js, .xml etc.
➔ It is strongly recommended to write Unit Tests(JUnit) whenever you write a piece of code or you
create a new file or make changes in existing code.
➔ It is not at all recommended to use -DskipTests=true or similar command which skips Unit Tests
while building ranger using maven command. Skipping unit tests is not advisable.
➔ If you are introducing a new functionality it is also recommended to write system tests(PyTests)
as a proof of verification of new functionality.
➔ If you are changing existing core business logic; it is recommended to run System Tests before
you commit or push your code to make sure that it’s not breaking the functionality.
➔ Clean code always Matters. It’s recommended to use camel case convention to keep code clean,

Ranger admin dev overview

More Related Content

What's hot

Viewers also liked

Similar to Ranger admin dev overview

Ranger admin dev overview