Running Kafka Node Docker Apps

2 min read 09-11-2024

Setting up a Kafka Node application using Docker can streamline your development process by providing an isolated environment that can be easily configured and managed. This guide will walk you through the essential steps to get your Kafka Node application up and running in a Docker container.

Prerequisites

Before you begin, ensure you have the following installed:

Docker: Make sure you have Docker installed on your machine. You can check this by running docker --version.
Docker Compose: This tool simplifies running multi-container Docker applications. You can verify its installation with docker-compose --version.

Step 1: Create a Project Directory

First, create a directory for your Kafka Node application:

mkdir kafka-node-app
cd kafka-node-app

Step 2: Create a Dockerfile

In the project directory, create a file named Dockerfile. This file will define your Node.js application's environment. Here's a simple example:

# Use the official Node.js image
FROM node:14

# Set the working directory
WORKDIR /usr/src/app

# Copy package.json and package-lock.json
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy the application code
COPY . .

# Expose the application port
EXPOSE 3000

# Command to run the app
CMD [ "node", "app.js" ]

Step 3: Create a docker-compose.yml File

Next, create a docker-compose.yml file to define your services, including Kafka and your Node.js application.

version: '3'

services:
  zookeeper:
    image: wurstmeister/zookeeper:3.4.6
    ports:
      - "2181:2181"

  kafka:
    image: wurstmeister/kafka:latest
    ports:
      - "9092:9092"
    environment:
      KAFKA_ZOOKEEPER: "zookeeper:2181"
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://localhost:9092
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT

  app:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - kafka

Step 4: Create Your Node.js Application

Now create a simple Node.js application. For example, create a file named app.js:

const express = require('express');
const Kafka = require('kafkajs').Kafka;

const app = express();
const port = 3000;

// Initialize Kafka
const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['kafka:9092']
});

// A simple endpoint to test
app.get('/', (req, res) => {
  res.send('Kafka Node App is running!');
});

// Start the application
app.listen(port, () => {
  console.log(`App listening at http://localhost:${port}`);
});

Don’t forget to create a package.json file to manage your dependencies:

{
  "name": "kafka-node-app",
  "version": "1.0.0",
  "main": "app.js",
  "dependencies": {
    "express": "^4.17.1",
    "kafkajs": "^1.15.0"
  }
}

Step 5: Build and Run Your Application

With your files in place, you can now build and run your application using Docker Compose:

docker-compose up --build

This command will build your Node.js application and start all defined services, including Zookeeper and Kafka.

Step 6: Access Your Application

Once the containers are running, you can access your Node.js application in your browser by navigating to:

http://localhost:3000

You should see a message indicating that your Kafka Node app is running.

Conclusion

You have successfully set up a Kafka Node application using Docker. This environment allows for easy scaling and management of your services. You can now build upon this foundation by integrating more complex functionalities with Kafka or expanding your Node.js application. Happy coding!