HTTP requests in YAML based pipelines

Getting started

When it comes to deployment pipelines, they come in all kinds of flavors. It could be a pipeline to deploy your new web application, updating your database with the latest tables and stored procedures you have created, or to deploy infrastructure into Azure to setup a new environment.

While there are a lot of use cases for deployment pipelines, and especially YAML-based pipelines, it might occur that you have to follow certain processes and that you are depending on external sources before you can continue your deployment or even be allowed to run it for specific environments, such as production.

When it comes to such external dependencies and if an API is available, we can manage this by using an HTTP task with wait for callback. This essentially allows us to wait for a specific reply from the external source before we continue with anything else in our pipeline.

Let’s check how this would work!

HTTP task

The HTTP task is an agentless task, which means it can’t do or be used for anything else. This also indicates that we can’t put any value that we receive in our callback into a runtime variable.

If this is something you do need, look at Powershell or Bash based solutions to make HTTP requests, but waiting on a specific callback will not be an option.

If we look at the REST API task, it has the following properties that can be used:

# Invoke a REST API as a part of your pipeline.
- task: InvokeRESTAPI@1
  inputs:
    #connectionType: 'connectedServiceName' # Options: connectedServiceName, connectedServiceNameARM
    #serviceConnection: # Required when connectionType == ConnectedServiceName
    #azureServiceConnection: # Required when connectionType == ConnectedServiceNameARM
    #method: 'POST' # Options: OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, PATCH
    #headers: 'Content-Type:application/json' 
    #body: # Required when method isn't GET or HEAD
    #urlSuffix: # Optional
    #waitForCompletion: 'false' # Options: true, false
    #successCriteria: # Optional

Service Connection

As you might have noticed, you don’t see any URL to which we can send a request. This is because of the service connection, in which we have 2 options: connectedServiceNameARM for requests towards Azure and the resources in it, and the generic option called connectedServiceName, allowing for any other services outside of Azure, for example a system used for handling approvals.

As it uses a Service Connection, you will need to set one up as such. In our case we will be using a generic service connection which requires us to add an URL to which we want to connect.

Properties

Now that this is done, we have a URL to work with, but now there are other properties left which are more common to HTTP requests, such as Method, Headers and the Body. It should be self-explanatory what you need to fill in, or at least be documented in the API you want to call to.

Last but not least ,we have the 3 remaining properties for URLsuffix, in which something can be added behind the base URL we specified in the Service Connection. This allows for more flexibility and it ensures that there don’t need to be multiple Service Connections for all the different options of the API you want to request something of.

WaitForCompletion is, as I mentioned, the important one. When true, it will wait for a specific callback. If false, it will work with the general response like status code 200 for success and expects the response to be received within 20 seconds.

And lastly, SuccessCriteria. In this you can add some conditions that might need to be checked.

Circling back to the WaitOnCompletion property: as stated, it will expect a callback to be made back to the Pipeline as to why it is on hold the whole time. But we would need to know where to callback, and what we need to add to make the callback pass successfully.

For this we need the following 2 things. Firstly, the CallbackURL, which is build up as follows: $(system.CollectionUri)$(system.TeamProjectId)/_apis/distributedtask/hubs/$(system.HostType)/plans/$(system.PlanId)/events?api-version=2.0-preview.1

Secondly, the CallbackBody, containing: { "name": "TaskCompleted", "taskId": $(system.TaskInstanceId), "jobId": $(system.JobId), "result": "succeeded" }

Example

Both can be parsed as JSON values in the Body of the REST API Task itself, so the external system can pick them up and use them at a later time. An example of an REST API Task would then be as follows:

- task: InvokeRESTAPI@1
  displayName: 'Invoke REST API to RemedyForce'
  inputs:
    connectionType: connectedServiceName
    serviceConnection: Remedyforce
    method: POST
    headers: |
      {
       "Content-Type":"application/json",
       "Authorization":"Bearer $(token)"
      }
    body: |
     {
     "callbackURL": "$(system.CollectionUri)$(system.TeamProjectId)/_apis/distributedtask/hubs/$(system.HostType)/plans/$(system.PlanId)/events?api-version=2.0-preview.1",
     "callbackBody": "{ "name": "TaskCompleted", "taskId": $(system.TaskInstanceId), "jobId": $(system.JobId), "result": "succeeded" }",
     "BMCServiceDesk__FKTemplate__c": "",
     "BMCServiceDesk__incidentDescription__c": ""
     }
    urlSuffix: 'BMCServiceDesk__Incident__c'
    waitForCompletion: 'true'

What's next?

Now that we know how to use the REST API Task in YAML, let's use this knowledge and see how we can apply it in other ways. Azure Pipelines have a default retainment rate of 30 days and we will be looking on how we can retain our pipelines per run with a HTTP request to Azure DevOps. Stay tuned!