New Job Scheduling Architecture

ELK (Elasticsearch, Logstash and Kibana) is an out-of-the-box 3^rd party tool that is used to store CloudShell logs. These logs can then be used to investigate deployment issues and bugs, and trace their causes.

Online test repository that contains tests of a specific type.

Identity Service is our authentication engine for the Job Scheduling Service and creates users based on Quali’s identity management mechanism, allowing these users to access and use the Job Scheduling Portal and REST API clients.

REST API-driven UI client that is used to interact with Job Scheduling Service. Job Scheduling Portal is separate from CloudShell Portal.

Job Scheduling Service is a high-performance test automation framework that allows organizations to create, manage and execute test batches. The service is hosted on a Kubernetes cluster and uses Quali Server to extract user, blueprint and sandbox data.

Open source database used to store New Job Scheduling-related data (suites, jobs, tests, reports, etc.) as well as permitted user identity information. MongoDB is installed as part of the Quali Server installation. There's also an option to use an existing MongoDB.

Open source message queuing tool used by Sandbox Service to manage sandbox creation requests coming from the Job Scheduling Portal and APIs. If needed, RabbitMQ is installed as part of the Quali Server.

Sandbox Service is a Python service that comes in two flavors: Windows service (embedded) delivered as part of Quali Server and Linux service included with the New Job Scheduling on Kubernetes (this second flavor supports extremely large deployments, up to hundreds of new sandboxes every few minutes). As such, we also recommend moving the Sandbox Service to the Kubernetes cluster option even for high load deployments without the New Job Scheduling.

This service performs two key functions:

• Processes, prioritizes and invokes all sandbox creation requests against Quali Server. As such, starting with CloudShell 2020.2, Sandbox Service replaces the reservation creation mechanism that was used in pre-2020.2 CloudShell versions, also in deployments that do not use the New Job Scheduling Service. The Windows flavor of the service includes a healthcheck mechanism that actively monitors and restarts the service in the event of a failure.
• Acts as the communication layer between Job Scheduling and Quali Server when handling Job Scheduling Service operations. In deployments that use Job Scheduling Services, the Kubernetes-based service needs to be used instead of the Sandbox Service that comes out-of-the-box with Quali Server. For details, see Modify CloudShell to Use the Kubernetes-based Services.

For upgrades in CloudShell environments that have Docker, Sandbox Service will be installed externally on Docker and the UseEmbeddedSandboxService configuration key needs to be used to configure the deployment to use this external Sandbox Service.

Test Discovery Service is responsible for discovering tests of a specific type. A Test Discovery Service can only discover tests of a specific type in a specific repository. For more information, see Test Discovery Service.

Test Execution Service is responsible for running tests discovered by the Test Discovery Service. For more information, see Test Execution Service.

Important: Test Execution Service does not handle shell and script commands. CloudShell Execution Servers are required to handle such operations (e.g. setup, teardown, resource commands and sandbox commands).