AWS Guide

• Introduction
  • Pre-requisites and Requirements
  • Architecture
• Planning Guide
  • Security
  • Costs
  • Sizing
• Deployment Guide
  • Steps
  • Loading Example Workflows
  • Install and Running Example Workflows
  • Adding a new user
  • Extra configuration for running PySpark
• S3 Integration
  • Installing aws cli
  • Configuring AWS access key and password
  • Access S3 in fire-ui
  • Protecting Data Using Server Side Encryption
  • REFERENCE : Creating Access Key & Secret Key
• Testing Fire Insights on AWS
  • Log into the System
  • View the Sample Applications
  • Execute a workflow on EMR
• Operational Guide
  • Onboarding New Users
  • Health Check
  • Backup and Recovery
  • Routing Maintenance
  • Support
• Copying files to S3 with aws-cli
  • Installing aws-cli on mac
  • Configure AWS Credentials
  • View S3 Buckets
  • View S3 Directory
  • Copy files to S3
  • Delete All Files in Directory
  • Setting Roles and Policies for EMR
  • REFERENCE : Creating Access Key & Secret Key
• Reading/Writing from S3
  • Dataset Processors
  • Reading from S3
  • Writing to S3
• Saving ML Model to S3
  • Saving Spark ML Model
  • Saving H20 ML Model
• Fire Integration with HIVE
  • Overview
  • Details
  • Writing to HIVE
• Fire Integration with Redshift
  • Redshift Processors
• Fire Integration with SageMaker
  • Spark Sagemaker Examples
  • Fire SageMaker Processors
  • AWS Provided Policies
  • Launching EMR
  • Create New Role
  • Use ARN of the new Role in the Workflow
  • AWS Instance Types
  • Dataset Column Names for Training with Sagemaker
  • Flow with Sparkflows and AWS
  • XGBoost Sagemaker Workflow
  • XGBoost Configuration
  • Executing the Workflow
• Fire Integration with Kinesis
  • Install AWS CLI
  • Create an access key and secret key
  • Configure AWS CLI
  • Create AWS Kinesis Stream
  • Send message to AWS Kinesis from AWS CLI
  • Update EMR_EC2_Default_Role
  • Or Create an IAM policy for accessing Amazon Kinesis
  • Create EMR Cluster with the above Role
  • Pushing data to Kinesis
  • Kinesis Workflow in Fire
  • REFERENCE : Creating Access Key & Secret Key
• File Watcher with AWS & Sparkflows
  • Overview
  • Design
  • Create an SQS Queue
  • Configure AWS S3 bucket to generate events
  • Create the AWS Lambda function
• CloudFormation Template with Embedded H2 DB
  • Overview
  • Relevant Files
  • Ports
  • Download Files and Upload to your S3 Bucket
  • Update Cloudformation template based on your environment
  • Steps to Create EMR Cluster and Deploy Fire
  • Connect Fire to the New Cluster
  • Load Examples
  • Create hadoop user
  • Start running the Examples
  • Summary
• CloudFormation Template with MySQL
  • Overview
  • Relevant Files
  • Ports
  • Download Files and Upload to your S3 Bucket
  • Update Cloudformation template based on your environment
  • Steps to Create EMR Cluster and Deploy Fire
  • Connect Fire to the New Cluster
  • Load Examples
  • Create hadoop user
  • Start running the Examples
  • Summary