# SNS Notification Configuration

**Date**: 2026-03-12  
**Status**: ✅ DOCUMENTED

---

## Overview

This document describes the SNS (Simple Notification Service) configuration used across all CircleCI pipelines for deployment notifications.

---

## SNS Topics

### Production SNS Topic

**Topic ARN**: `arn:aws:sns:ca-central-1:489176460030:DevOps-alert`
- **Region**: ca-central-1
- **AWS Account**: 489176460030 (Production)
- **Topic Name**: DevOps-alert

**Used By**:
- All 29 production services (prod/ and redtag-prod/ folders)
- 20 ACC production services
- 7 other production services
- 2 redtag-prod services

### Development SNS Topic

**Topic ARN**: `arn:aws:sns:ca-central-1:594765360358:tbi-devops-secrets-pipeline-notices`
- **Region**: ca-central-1
- **AWS Account**: 594765360358 (Development)
- **Topic Name**: tbi-devops-secrets-pipeline-notices

**Used By**:
- dev/tbi-package-travelbrands
- dev/tbi-pl-dynamic

**Note**: Most dev services do not currently send SNS notifications.

---

## Notification Types

### Production Services (4 notifications per deployment)

1. **BUILD Approval Needed**
   - Sent after tests pass
   - Requires manual approval to proceed with Docker build
   - Subject: `🔒 PROD Approval Needed: BUILD - [service-name] #[build-num]`

2. **DEPLOY Approval Needed**
   - Sent after Docker image is built and pushed to ECR
   - Requires manual approval to deploy to production
   - Subject: `🔒 PROD Approval Needed: DEPLOY - [service-name] #[build-num]`

3. **Deployment Success**
   - Sent after successful deployment to Kubernetes
   - Subject: `DevOps CI/CD Pipeline Notification`
   - Message starts with: `✅ Deployment Successful`

4. **Deployment Failure**
   - Sent when deployment fails (on_fail condition)
   - Subject: `DevOps CI/CD Pipeline Notification`
   - Message starts with: `❌ Deployment Failed`

### Development Services (2 notifications per deployment)

1. **Deployment Success**
2. **Deployment Failure**

---

## Message Format

### BUILD Approval Notification
```
⏳ PRODUCTION Pipeline - Awaiting BUILD Approval

Service: [service-name]
Environment: PRODUCTION
Build: #[build-number]
Branch: [branch-name]
Commit: [commit-sha]
Image Tag: [version]
Triggered by: [username]

✅ Tests have passed successfully

⏳ MANUAL APPROVAL REQUIRED to proceed with Docker image build

To approve:
1. Go to: [CircleCI build URL]
2. Click "Approve" on the approve-build job

Once approved, the Docker image will be built and pushed to ECR.
```

### DEPLOY Approval Notification
```
⏳ PRODUCTION Pipeline - Awaiting DEPLOY Approval

Service: [service-name]
Environment: PRODUCTION
Namespace: [namespace]
Build: #[build-number]
Branch: [branch-name]
Commit: [commit-sha]
Image Tag: [version]
Triggered by: [username]

✅ Tests passed
✅ Docker image built and pushed to ECR

⏳ MANUAL APPROVAL REQUIRED to deploy to PRODUCTION cluster

To approve:
1. Go to: [CircleCI build URL]
2. Click "Approve" on the approve-deploy job

⚠️  WARNING: This will deploy to PRODUCTION Kubernetes cluster!
⚠️  Ensure the image has been tested and validated.

Once approved, the service will be deployed to production.
```

### Success Notification
```
✅ Deployment Successful - [service-name]

Environment: [environment]
Namespace: [namespace]
Image Tag: [version]
Build: #[build-number]
Branch: [branch-name]
Commit: [commit-sha]
Triggered by: [username]

View build: [CircleCI build URL]
```

### Failure Notification
```
❌ Deployment Failed - [service-name]

Environment: [environment]
Namespace: [namespace]
Image Tag: [version]
Build: #[build-number]
Branch: [branch-name]
Commit: [commit-sha]
Triggered by: [username]

View build: [CircleCI build URL]

Please check the build logs for error details.
```

---

## Implementation in CircleCI

### Notification Jobs

Each production service has dedicated notification jobs:

```yaml
notify-await-build-approval:
  executor: aws-cli/default
  resource_class: small
  steps:
    - checkout
    - aws-cli/setup:
        profile_name: default
    - run:
        name: Send SNS notification - Awaiting BUILD approval
        command: |
          # Create message file
          cat > /tmp/sns-approval-needed.txt << EOF
          [message content]
          EOF
          
          # Publish to SNS
          aws sns publish \
            --region ca-central-1 \
            --topic-arn "arn:aws:sns:ca-central-1:489176460030:DevOps-alert" \
            --subject "[subject]" \
            --message file:///tmp/sns-approval-needed.txt
```

### Deployment Notifications

Success and failure notifications are embedded in the `deploy-to-kubernetes` job:

```yaml
- run:
    name: Send deployment success notification
    command: |
      aws sns publish \
        --region ca-central-1 \
        --topic-arn "arn:aws:sns:ca-central-1:489176460030:DevOps-alert" \
        --subject "DevOps CI/CD Pipeline Notification" \
        --message file:///tmp/sns-message.txt

- run:
    name: Send deployment failure notification
    when: on_fail
    command: |
      aws sns publish \
        --region ca-central-1 \
        --topic-arn "arn:aws:sns:ca-central-1:489176460030:DevOps-alert" \
        --subject "DevOps CI/CD Pipeline Notification" \
        --message file:///tmp/sns-failure.txt
```

---

## Teams Integration (Future Enhancement)

To send SNS notifications to Microsoft Teams channels, you need:

### 1. Teams Incoming Webhook
- Create an incoming webhook in your Teams channel
- Get the webhook URL: `https://outlook.office.com/webhook/...`

### 2. AWS Lambda Function
Create a Lambda function that:
- Subscribes to the SNS topic
- Receives SNS messages
- Formats them for Teams (Adaptive Cards format)
- Posts to Teams webhook URL

### 3. Lambda Configuration
- **Runtime**: Python 3.x or Node.js
- **Environment Variables**: Teams webhook URL
- **IAM Role**: SNS read permissions
- **Trigger**: Subscribe Lambda to SNS topic

### 4. Message Formatting
Transform SNS messages into Teams Adaptive Cards with:
- Color coding (green=success, red=failure, yellow=approval)
- Action buttons for CircleCI links
- Structured layout with sections
- Emoji indicators

---

## Current Subscriptions

### Production SNS Topic Subscriptions
To view current subscriptions:
```bash
aws sns list-subscriptions-by-topic \
  --topic-arn "arn:aws:sns:ca-central-1:489176460030:DevOps-alert" \
  --region ca-central-1
```

### Development SNS Topic Subscriptions
To view current subscriptions:
```bash
aws sns list-subscriptions-by-topic \
  --topic-arn "arn:aws:sns:ca-central-1:594765360358:tbi-devops-secrets-pipeline-notices" \
  --region ca-central-1
```

---

## Troubleshooting

### No Notifications Received

1. **Check SNS Topic Permissions**
   ```bash
   aws sns get-topic-attributes \
     --topic-arn "arn:aws:sns:ca-central-1:489176460030:DevOps-alert" \
     --region ca-central-1
   ```

2. **Verify CircleCI Context**
   - Ensure `devops-mgt-prod` context has AWS credentials
   - Check AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

3. **Check SNS Publish Logs**
   - Review CircleCI build logs for SNS publish errors
   - Look for AWS CLI error messages

4. **Verify Subscriptions**
   - Ensure email/endpoint is subscribed to the topic
   - Check subscription confirmation status

### Notification Delays

- SNS typically delivers within seconds
- Check AWS CloudWatch metrics for SNS topic
- Review Lambda function logs if using Teams integration

---

## Security Considerations

1. **IAM Permissions**
   - CircleCI requires `sns:Publish` permission
   - Use least-privilege IAM policies
   - Rotate AWS credentials regularly

2. **Topic Access**
   - Restrict SNS topic access to authorized services
   - Use resource-based policies
   - Enable CloudTrail logging for SNS

3. **Message Content**
   - Avoid including sensitive data in notifications
   - Use commit SHAs instead of full commit messages
   - Sanitize user inputs

---

## References

- **AWS SNS Documentation**: https://docs.aws.amazon.com/sns/
- **CircleCI AWS CLI Orb**: https://circleci.com/developer/orbs/orb/circleci/aws-cli
- **Teams Incoming Webhooks**: https://docs.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/

---

**Last Updated**: 2026-03-12  
**Maintained By**: DevOps Team