Axual Platform Standalone

Overview

The standalone is a simple server based on the Axual Platform, providing a testbed for developers. Using it you can set up a single or multi-cluster instance, against which you test your applications on integration level.

Features

  • Single and multi-cluster support

  • Possible to use own keystores for servers

  • Fully compatible with Axual Client

  • Supports advanced ACL formatting

  • Avro Support (Avro JAR must be added to classpath when started)

  • Define new application IDs using the Rest API

  • Define new stream definitions using the Rest API

  • View cluster assignments using the Rest API

  • Assign application IDs to different cluster using the Rest API

  • Change TTL, Distributor Timeout and Distance using the Rest API

Limitations

  • Discovery API is mocked

  • Distributor distance and timeout is static for all applications

  • TTL is static

  • Schema Registry is mocked, only basic SerDe calls needed for runtime supported

  • No ACLs support (for ACL, add them using AdminClient)

Run Standalone (JAR)

  1. Download the Axual Platform Standalone fat JAR from the repository.

  2. Create an application.yml (see Standalone YAML Examples)

  3. Run with java -Dserver.path=8080 -jar for normal running.

  4. To run with extra classpath entries, run with java -Dserver.path=8080 -Dloader.path <extra classpaths, comma separated> -jar

Example configurations

The following configuration examples are for both single cluster and multi-cluster run.

Single Cluster

  • Tenant: axual

  • Instance: platform-standalone

  • Default environment: local

  • Discovery API TTL: 5000 milliseconds

  • Discovery API Distributor Distance: 2

  • Discovery API Distributor Timeout: 20000 milliseconds

  • Bind address: 0.0.0.0

  • Advertised address: localhost

  • Rest API port: 8080

  • Discovery API port: 8081

  • Cluster

    • Name: clusterA

    • Schema Registry port: 8082

    • Zookeeper port: 8083

    • Broker port: 8084

    • Use Advanced ACL: true

    • Topic Pattern: {tenant}-{instance}-{environment}-{topic}

    • Group Id Pattern: {tenant}-{instance}-{environment}-{group}

  • Application 1

    • Application ID: io.axual.test-1

    • Environment: local

  • Application 2

    • Application ID: io.axual.test-2

    • Environment: local2

  • Stream 1

    • Name: general-test

    • Environment: local

    • Is raw topic: false

    • Avro Class Key: io.axual.client.test.Random

    • Avro Class Value: io.axual.client.test.Random

  • Stream 2

    • Name: general-test

    • Environment: local2

    • Is raw topic: false

    • Avro Class Key: io.axual.client.test.Random

    • Avro Class Value: io.axual.client.test.Random

Multi-Cluster (Dual-Cluster)

  • Tenant: axual

  • Instance: platform-standalone

  • Default environment: local

  • Discovery API TTL: 5000 milliseconds

  • Discovery API Distributor Distance: 2

  • Discovery API Distributor Timeout: 20000 milliseconds

  • Bind address: 0.0.0.0

  • Advertised address: localhost

  • Rest API port: 8080

  • Discovery API port: 8081

  • Cluster 1

    • Name: clusterA

    • Schema Registry port: 8082

    • Zookeeper port: 8083

    • Broker port: 8084

    • Use Advanced ACL: true

    • Topic Pattern: {tenant}-{instance}-{environment}-{topic}

    • Group Id Pattern: {tenant}-{instance}-{environment}-{group}

  • Cluster 2

    • Name: clusterB

    • Schema Registry port: 8092

    • Zookeeper port: 8093

    • Broker port: 8094

    • Use Advanced ACL: true

    • Topic Pattern: {tenant}-{instance}-{environment}-{topic}

    • Group Id Pattern: {tenant}-{instance}-{environment}-{group}

  • Application 1

    • Application ID: io.axual.test-1

    • Environment: local

  • Application 2

    • Application ID: io.axual.test-2

    • Environment: local2

  • Stream 1

    • Name: general-test

    • Environment: local

    • Is raw topic: false

    • Avro Class Key: io.axual.client.test.Random

    • Avro Class Value: io.axual.client.test.Random

  • Stream 2

    • Name: general-test

    • Environment: local2

    • Is raw topic: false

    • Avro Class Key: io.axual.client.test.Random

    • Avro Class Value: io.axual.client.test.Random

Standalone YAML Examples

Single Cluster Standalone YAML Example

standalone:
    type: single
    tenant: axual
    instance: platform-standalone
    environment: local
    ttl: 5000
    distributorTimeout: 20000
    distributorDistance: 2
    bindAddress: 0.0.0.0
    advertisedAddress: localhost
    clusterName: clusterA
    endpointPort: 8081
    schemaRegistryPort: 8082
    zookeeperPort: 8083
    brokerPort: 8084
    useAdvancedAcl: true
    topicPattern: '{tenant}-{instance}-{environment}-{topic}'
    groupIdPattern: '{tenant}-{instance}-{environment}-{group}'
    applications:
        0:
            applicationId: io.axual.test-1
            environment: local
        1:
            applicationId: io.axual.test-1
            environment: local2
    sslConfig:
        keystoreLocation: ~/standalone.server.keystore.jks
        keystorePassword: notsecret
        keyPassword: notsecret
        truststoreLocation: ~/standalone.server.truststore.jks
        truststorePassword: notsecret
        hostnameVerificationEnabled: false
    streams:
        0:
            name: general-test
            isRawTopic: false
            keyClass: io.axual.client.test.Random
            valueClass: io.axual.client.test.Random
            partitions: 2
            environment: local
        1:
            name: general-test
            isRawTopic: false
            keyClass: io.axual.client.test.Random
            valueClass: io.axual.client.test.Random
            partitions: 2
            environment: local2

Multi-Cluster Standalone YAML Example

standalone:
  type: multi
  tenant: axual
  instance: platform-standalone
  environment: local
  ttl: 5000
  distributorTimeout: 20000
  distributorDistance: 2
  bindAddress: 0.0.0.0
  advertisedAddress: localhost
  endpointPort: 8081
  enableDistribution: true
  clusterSettings:
    0:
      name: clusterA
      schemaRegistryPort: 8082
      zookeeperPort: 8083
      brokerPort: 8084
      useAdvancedAcl: true
      topicPattern: '{tenant}-{instance}-{environment}-{topic}'
      groupIdPattern: '{tenant}-{instance}-{environment}-{group}'
    1:
      name: clusterB
      schemaRegistryPort: 8092
      zookeeperPort: 8093
      brokerPort: 8094
      useAdvancedAcl: false
      topicPattern: '{tenant}-{instance}-{environment}-{topic}'
      groupIdPattern: '{tenant}-{instance}-{environment}-{group}'
  applications:
    0:
      applicationId: io.axual.test-1
      environment: local
      assignedCluster: clusterA
    1:
      applicationId: io.axual.test-1
      environment: local2
      assignedCluster: clusterB
  sslConfig:
    keystoreLocation: ~/standalone.server.keystore.jks
    keystorePassword: notsecret
    keyPassword: notsecret
    truststoreLocation: ~/standalone.server.truststore.jks
    truststorePassword: notsecret
    hostnameVerificationEnabled: false
  streams:
    0:
      name: general-test
      isRawTopic: false
      keyClass: io.axual.client.test.Random
      valueClass: io.axual.client.test.Random
      partitions: 2
      environment: local
    1:
      name: general-test
      isRawTopic: false
      keyClass: io.axual.client.test.Random
      valueClass: io.axual.client.test.Random
      partitions: 2
      environment: local2

Run Single Cluster Standalone (Docker)

docker run -ti --rm --name axual-standalone \
    -p 8080-8081:8080-8081 \
    -p 8082-8084:8082-8084 \
    -e SERVER_PORT=8080 \
    -e STANDALONE_TYPE=single \
    -e STANDALONE_TENANT=axual \
    -e STANDALONE_INSTANCE=platform-standalone \
    -e STANDALONE_ENVIRONMENT=local \
    -e STANDALONE_TTL=1000 \
    -e STANDALONE_DISTRIBUTOR_DISTANCE=2 \
    -e STANDALONE_DISTRIBUTOR_TIMEOUT=20000 \
    -e STANDALONE_BIND_ADDRESS=0.0.0.0 \
    -e STANDALONE_ADVERTISED_ADDRESS=localhost \
    -e STANDALONE_CLUSTER_NAME=clusterA \
    -e STANDALONE_ENDPOINT_PORT=8081 \
    -e STANDALONE_SCHEMA_REGISTRY_PORT=8082 \
    -e STANDALONE_ZOOKEEPER_PORT=8083 \
    -e STANDALONE_BROKER_PORT=8084 \
    -e STANDALONE_USE_ADVANCED_ACL=true \
    -e STANDALONE_TOPIC_PATTERN='{tenant}-{instance}-{environment}-{topic}' \
    -e STANDALONE_GROUP_ID_PATTERN='{tenant}-{instance}-{environment}-{group}' \
    -e STANDALONE_APPLICATIONS_0_APPLICATION_ID=io.axual.test-1 \
    -e STANDALONE_APPLICATIONS_0_ENVIRONMENT=local \
    -e STANDALONE_APPLICATIONS_1_APPLICATION_ID=io.axual.test-2 \
    -e STANDALONE_APPLICATIONS_1_ENVIRONMENT=local2 \
    -e STANDALONE_STREAMS_0_NAME=general-test \
    -e STANDALONE_STREAMS_0_IS_RAW_TOPIC=false \
    -e STANDALONE_STREAMS_0_KEY_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_0_VALUE_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_0_PARTITIONS=2 \
    -e STANDALONE_STREAMS_0_ENVIRONMENT=local \
    -e STANDALONE_STREAMS_1_NAME=general-test \
    -e STANDALONE_STREAMS_1_IS_RAW_TOPIC=false \
    -e STANDALONE_STREAMS_1_KEY_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_1_VALUE_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_1_PARTITIONS=2 \
    -e STANDALONE_STREAMS_1_ENVIRONMENT=local2 \
    docker.axual.io/axaul/platform-test-standalone:5.4.1

Run Multi Cluster Standalone (Docker)

docker run -ti --rm --name axual-standalone \
    -p 8080-8081:8080-8081 \
    -p 8082-8084:8082-8084 \
    -p 8092-8094:8092-8094 \
    -e SERVER_PORT=8080 \
    -e STANDALONE_TYPE=multi \
    -e STANDALONE_TENANT=axual \
    -e STANDALONE_INSTANCE=platform-standalone \
    -e STANDALONE_ENVIRONMENT=local \
    -e STANDALONE_TTL=1000 \
    -e STANDALONE_DISTRIBUTOR_DISTANCE=2 \
    -e STANDALONE_DISTRIBUTOR_TIMEOUT=20000 \
    -e STANDALONE_BIND_ADDRESS=0.0.0.0 \
    -e STANDALONE_ADVERTISED_ADDRESS=localhost \
    -e STANDALONE_ENDPOINT_PORT=8081 \
    -e STANDALONE_USE_ADVANCED_ACL=true \
    -e STANDALONE_ENABLE_DISTRIBUTION=true \
    -e STANDALONE_CLUSTERS_0_NAME=clusterA \
    -e STANDALONE_CLUSTERS_0_SCHEMA_REGISTRY_PORT=8082 \
    -e STANDALONE_CLUSTERS_0_ZOOKEEPER_PORT=8083 \
    -e STANDALONE_CLUSTERS_0_BROKER_PORT=8084 \
    -e STANDALONE_CLUSTERS_0_TOPIC_PATTERN='{tenant}-{instance}-{environment}-{topic}' \
    -e STANDALONE_CLUSTERS_0_GROUP_ID_PATTERN='{tenant}-{instance}-{environment}-{group}' \
    -e STANDALONE_CLUSTERS_1_NAME=clusterB \
    -e STANDALONE_CLUSTERS_1_SCHEMA_REGISTRY_PORT=8092 \
    -e STANDALONE_CLUSTERS_1_ZOOKEEPER_PORT=8093 \
    -e STANDALONE_CLUSTERS_1_BROKER_PORT=8094 \
    -e STANDALONE_CLUSTERS_1_TOPIC_PATTERN='{tenant}-{instance}-{environment}-{topic}' \
    -e STANDALONE_CLUSTERS_1_GROUP_ID_PATTERN='{tenant}-{instance}-{environment}-{group}' \
    -e STANDALONE_APPLICATIONS_0_APPLICATION_ID=io.axual.test-1 \
    -e STANDALONE_APPLICATIONS_0_ENVIRONMENT=local \
    -e STANDALONE_APPLICATIONS_0_ASSIGNED_CLUSTER=clusterA \
    -e STANDALONE_APPLICATIONS_1_APPLICATION_ID=io.axual.test-2 \
    -e STANDALONE_APPLICATIONS_1_ENVIRONMENT=local2 \
    -e STANDALONE_APPLICATIONS_1_ASSIGNED_CLUSTER=clusterB \
    -e STANDALONE_STREAMS_0_NAME=general-test \
    -e STANDALONE_STREAMS_0_IS_RAW_TOPIC=false \
    -e STANDALONE_STREAMS_0_KEY_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_0_VALUE_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_0_PARTITIONS=2 \
    -e STANDALONE_STREAMS_0_ENVIRONMENT=local \
    -e STANDALONE_STREAMS_1_NAME=general-test \
    -e STANDALONE_STREAMS_1_IS_RAW_TOPIC=false \
    -e STANDALONE_STREAMS_1_KEY_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_1_VALUE_CLASS=io.axual.client.test.Random \
    -e STANDALONE_STREAMS_1_PARTITIONS=2 \
    -e STANDALONE_STREAMS_1_ENVIRONMENT=local2 \
    docker.axual.io/axaul/platform-test-standalone:5.4.1

Standalone Rest API

The Standalone Rest API can be used to view and alter the Standalone configurations. The port where the API listens is defined by the environment variable SERVER_PORT or Java argument server.port. All are HTTP GET operations enabling developers to use a web browser for the calls.

Instance operations

Instance operations control the system at an instance level, for all registered applications

Get instance configuration

URL: /standalone/instance
Verb: GET
Returns the Instance configuration

Get TTL

URL: /standalone/ttl
Verb: GET
Returns the TTL for this instance

Set TTL

URL: /standalone/ttl/set?value=XXX
Verb: GET
Input: value - The TTL in milliseconds
Sets the TTL to a new value for all applications in the instance

Get Distributor Distance

URL: /standalone/distributor/distance
Verb: GET
Returns the distributor distance for this instance

Set Distributor Distance

URL: /standalone/distributor/distance/set?value=XXX
Verb: GET
Input: value - The new distributor distance in steps Sets the distributor distance to a new value for all applications in the instance

Get Distributor Timeout

URL: /standalone/distributor/timeout
Verb: GET
Returns the distributor timeout for this instance

Set Distributor Timeout

URL: /standalone/distributor/timeout/set?value=XXX
Verb: GET
Input: value - The new distributor timeout in steps
Sets the distributor timeout to a new value for all applications in the instance

Cluster operations

Cluster operations control the standalone platform on a cluster level.

Get available cluster names

URL: /standalone/cluster
Verb: GET
Returns a list of cluster names

Get cluster configuration

URL: /standalone/cluster/{cluster-name}
Verb: GET
Input: clustername - The name of the cluster
Returns the cluster configuration

Application operations

The application operations will give you information about application registrations within the standalone platform

Get all applications

URL: /standalone/application
Verb: GET
Returns a list of application configurations

Create an application

URL: /standalone/application/create/{applicationid}?environment=XXX&clusterName=YYY
Verb: GET
Input: applicationId - The application ID of this new application registration
Input: environment - Specifies for which environment the application registration is. If not set then the environment set in the instance configuration will be used.
Input: clusterName - Specifies to which cluster this application registration should be assigned. If not set then the first available cluster definition will be used.
Returns the application registration data

Stream operations

The stream operations will give you information about stream registrations within the standalone platform

Get all streams

URL: /standalone/stream
Verb: GET
Returns a Map of stream names and the corresponding stream registration.

Create a stream

URL: /standalone/application/stream/{name}?isRaw=true/&partitions=5&keyClass=io.axual.client.test.Random&valueClass=io.axual.client.test.Random
Verb: GET
Input: name - the stream name
Input: isRaw - a boolean flag (true/false), no name resolution will be used when true, creating a topic with the specified name. When false or not set the topic pattern will be used to create the topic
Input: environment - Specifies for which environment the application registration is. If not set then the environment set in the instance configuration will be used.
Input: partitions - Specifies to number of partitions to create for this stream. Default value is 3.
Input: keyClass - The name of the Avro Schema Compiled Classname to use for the key. Can be left unset if Avro is not used.
Input: valueClass - The name of the Avro Schema Compiled Classname to use for the value. Can be left unset if Avro is not used.
Returns the stream registration data.

Assignment operations

Controls the assignments of applications to clusters.

Get all assignments

URL: /standalone/assignment
Returns a Map of Cluster Names with a list of application registrations assigned to that cluster.

Get all assignments for a cluster

URL: /standalone/assignment/{cluster-name}
Verb: GET
Input: clustername - The name of the cluster
Returns List of application registrations assigned to the cluster.

Assign an application to a cluster

URL: /standalone/assignment/{cluster-name}/set/{applicationid}?environment=XXX
Verb: GET
Input: clustername - The name of the cluster
Input: applicationId - The id of the application registration to assign to the cluster
Input: environment - Specifies for which environment the application registration is. If not set then the environment set in the instance configuration will be used. Returns the new application registration.