Multimodal Data Processing - BDA

A production-grade multimodal knowledge system built with AWS CDK, Amazon Bedrock, and React. This application enables intelligent processing and querying of various media types including documents, audio, and video through a RAG (Retrieval-Augmented Generation) architecture.

Architecture Overview

• Backend: AWS CDK infrastructure with Lambda functions, API Gateway, Cognito, DynamoDB, and Amazon Bedrock Knowledge Base
• Frontend: React + TypeScript application with Vite, Tailwind CSS, and AWS Amplify integration
• Storage: S3 buckets for input/output media and processed documents
• AI/ML: Amazon Bedrock for document processing and Q&A capabilities

Prerequisites

Before running this project, ensure you have the following installed:

Required

• Node.js (v18 or higher)
• Python (v3.9 or higher)
• AWS CLI (v2) - Installation Guide
• AWS CDK - Install globally: npm install -g aws-cdk
• Git (for version control)

Optional

• Git Bash or WSL (for Windows users to run bash scripts)
• jq (for JSON parsing in bash scripts) - Installation Guide

AWS Configuration

• AWS account with appropriate permissions
• AWS CLI configured with credentials: aws configure
• Access to Amazon Bedrock services in your AWS region

Project Structure

multimodal-data-processing-bda/
├── backend/                      # AWS CDK Infrastructure
│   ├── stack/                    # CDK stack definitions
│   │   ├── stack_constructs/     # Reusable AWS constructs
│   │   └── lambdas/              # Lambda function code
│   ├── app.py                    # CDK app entry point
│   ├── requirements.txt          # Python dependencies
│   └── source.bat                # Windows venv activation helper
│
└── talk-to-any-media-front-end/  # React Frontend
    ├── src/                      # Source code
    ├── components/               # React components
    ├── generate-env.sh           # Environment generation script
    ├── package.json              # Node.js dependencies
    └── vite.config.ts            # Vite configuration

Installation & Setup

1. Clone the Repository

git clone <repository-url>
cd multimodal-data-processing-bda

2. Backend Setup

Install Python Dependencies

cd backend

# Create and activate virtual environment
python -m venv .venv

# On Windows
.venv\Scripts\activate
# or use the helper script:
source.bat

# On Linux/Mac
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

Bootstrap AWS CDK (First Time Only)

# Bootstrap CDK in your AWS account/region
cdk bootstrap

Deploy Backend Infrastructure

# Synthesize CloudFormation template
cdk synth

# Deploy the stack
cdk deploy

# Approve IAM changes when prompted

Note: The deployment will create:

• Cognito User Pool and Identity Pool for authentication
• API Gateway REST API
• Lambda functions for media processing and Q&A
• S3 buckets for input/output storage
• DynamoDB table for job tracking
• Amazon Bedrock Knowledge Base

3. Frontend Setup

Install Node.js Dependencies

cd ../talk-to-any-media-front-end
npm install

Generate Environment Configuration

After the backend is deployed, generate the frontend .env file:

On Windows (PowerShell - Recommended):

.\generate-env.ps1

On Windows (Git Bash):

bash generate-env.sh

On Linux/Mac:

chmod +x generate-env.sh
./generate-env.sh

This script will:

1. Fetch CloudFormation stack outputs
2. Extract required configuration values
3. Generate .env file with all necessary environment variables

Manual Configuration (Alternative):

If the script doesn't work, manually create a .env file:

VITE_REGION_NAME=<your-aws-region>
VITE_COGNITO_USER_POOL_ID=<from-stack-outputs>
VITE_COGNITO_USER_POOL_CLIENT_ID=<from-stack-outputs>
VITE_COGNITO_IDENTITY_POOL_ID=<from-stack-outputs>
VITE_API_GATEWAY_REST_API_ENDPOINT=<from-stack-outputs>
VITE_APP_NAME="Bedrock BDA Multimodal Media Solution"
VITE_S3_BUCKET_NAME=<from-stack-outputs>

Get the values from AWS Console > CloudFormation > AnyMedia-RAG-Backend-Stack > Outputs tab.

Running the Application

Start the Frontend Development Server

cd talk-to-any-media-front-end
npm run dev

The application will be available at http://localhost:5173

First Time Usage

1. Create an Account: Use the sign-up flow to create a Cognito user
2. Verify Email: Check your email for the verification code
3. Sign In: Log in with your credentials
4. Upload Media: Start uploading documents, audio, or video files
5. Ask Questions: Use the Q&A interface to query your uploaded content

Development Commands

Backend (CDK)

cd backend

# Activate virtual environment
source .venv/bin/activate  # Linux/Mac
.venv\Scripts\activate     # Windows

# Synthesize CloudFormation template
cdk synth

# Show differences between deployed and local
cdk diff

# Deploy infrastructure changes
cdk deploy

# Destroy all resources (WARNING: Deletes everything)
cdk destroy

Frontend

cd talk-to-any-media-front-end

# Development server with hot reload
npm run dev

# Build for production
npm run build

# Preview production build
npm run preview

# Regenerate environment variables (PowerShell)
.\generate-env.ps1

# Or using bash
bash generate-env.sh

Testing

Test Backend API

A test script is provided to verify backend functionality:

cd talk-to-any-media-front-end
node test-backend.cjs

Deployment to Production

Build Frontend for Production

cd talk-to-any-media-front-end
npm run build

The production build will be in the dist/ directory. Deploy this to:

• AWS S3 + CloudFront
• AWS Amplify Hosting
• Any static hosting service

Backend Production Deployment

The CDK stack is production-ready. For additional environments:

1. Create separate CDK stacks for dev/staging/prod
2. Use CDK context variables for environment-specific configuration
3. Implement CI/CD with AWS CodePipeline or GitHub Actions

Troubleshooting

Common Issues

AWS CLI not configured

aws configure
# Enter your AWS Access Key ID, Secret Access Key, and default region

CDK deployment fails

• Ensure you have sufficient IAM permissions
• Check AWS service quotas (especially Bedrock availability)
• Review CloudFormation events in AWS Console

Frontend can't connect to backend

• Verify .env file has correct values
• Check CORS settings in API Gateway
• Ensure Cognito user pool is properly configured

Environment generation script fails

• Ensure AWS CLI is installed and configured
• Verify the stack name matches: AnyMedia-RAG-Backend-Stack
• Check you have permissions to read CloudFormation outputs
• On Windows: Use PowerShell script .\generate-env.ps1 (no additional dependencies needed)
• On Linux/Mac with bash: Ensure jq is installed for JSON parsing

Security Best Practices

• Never commit .env files (already in .gitignore)
• Rotate AWS credentials regularly
• Use least-privilege IAM policies
• Enable MFA for AWS accounts
• Review CDK Nag security findings before deployment
• Keep dependencies updated (npm audit, pip check)

Project Tags

• Project: AnyMediaRAG
• Environment: Development

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request

License

[Add your license here]

Support

For issues and questions:

• Create an issue in the repository
• Check AWS documentation for service-specific questions
• Review CloudFormation stack events for deployment issues