Trigger webhook in BPM?

[Alex8000]

Hi all,

We are looking for a way to integrate our (JSON enabled) work slip application within our workflow. Is it possible to have the BPM either trigger a preconfigured Webhook or have it create the relevant API call on the fly? There is a JIRA integration node within the BPM, but nothing more generic for us to use as far as I know.

Thanks!

Best regards,

Alex

[Martyn Houghton]

@Alex8000

This it something we are looking to use for a file provisioning service, with the BPM triggering the file secure file transfer process in response to Service Request.

I thought this was already part of the system, but as you state it is not currently present, with web hooks only being able to the triggered on application events, which is quite a bit of a limitation. 

Cheers

Martyn

[Gerry]

@Martyn Houghton  @Alex8000

WebHooks are a system for allowing the Hornbill platform to call out to other applications on certain application events, they are not designed or intended to be a general API definition that can be used from within BPM, so I would like to think that what you are wanting to do is not a limitation of WebHooks.  If I understand what you are requiring is the ability of the Hornbill BPM to be able to orchestrate actions for other external tools/systems.  

The ideal solution to this requirement does not have anything to do with WebHooks, what would work is to have a node type in the BPM tool that allows you to make a web call to another external application and this is something we did consider.  However, our design intention with the BPM was to keep it so "non-programmers" (aka business process analysts) could use it without any programming knowledge. and the problem with adding something like a generic API calling node is that there would be no escaping the need to have programming skills to map request params and possibly return results back from the called application into the BPM for onward processing.  This I felt was taking the BPM too far towards a development environment getting us back to something that was non-upgradable like we had with Supportworks and VPME customisations.  I believe as a business process orchestration tool we have found the right balance and and kept the BPM a process tool and prevented it from becoming a programming tool. 

Now I appreciate that for those users of Hornbill that are more technical this is not good news, and I understand that completely.  To implement something that meets your requirement properly there needs to be a complete separation of the "programming" element from the use of the BPM, something that can be used to create (and program up as required) an integration into other tools and exposing that to the BPM in a BPM-user-friendly way.  To implement this properly there is an approach to integration known as Run Book Automation.  To better understand what that is I should describe FlowCode - which is what we have used to provide the pre-packaged integration to Jira which Alex mentions.  FlowCode is a lot like VPME from Supportworks, in fact its VPME evolved and broadly based on the same concepts.  FlowCode it the environment that our application dev teams use to create server-side functional logic, and its the FlowCode layer that makes it possible for application devs to expose application functions to the BPM and to the API endpoints.

So to properly implement that what we need to provide is a Runbook Automation Layer/Application/Tool similar to FlowCode where customers can create their own micro services for their instance which can be exposed into scope of the BPM for use within process orchestration.  This is something thats been in my minds eye in our platform roadmap since the inception of the BPM design for Hornbill but its not something we have gotten around to so far.  Ideally customers should not have to "code" for any integration, integrations should be pre-build packages that you can just select and use - thats the design indentation of the Hornbill platform and the approach we took with the Jira integration which is why its available in the BPM, and no programming is required.

Implementing these kinds of integrations is not-trivial, it can be as simple as a HTTP or HTTPS call, but more often than not its more than that, you need to handle request and response params, sometimes complex or large amounts of data, you need to handle security issues like authentication and network routing (for the case of cloud to internal apps) and you need to pay a lot of attention to error conditions and error handling for a myriad of transport, application and security error conditions, not only to run such things but to debug when problems arise.  

It is technically possible to provide a simple web call node in BPM but I am really not comfortable with that as a solution, its something that in theory is quite simple to add. but once we add it, customers will use it and then we can never remove it without breaking customers workflows, so the purity of the design intent for BPM being an non-programming environment will be broken forever.

Perhaps we could get some more specifics around the integrations you are trying to get? Are these standard web applications? It might be possible for us to build the integrations like we have done already for Jira...

 

Gerry


 

[Alex8000]

Hi @Gerry,
 

I appreciate you taking the time to respond to this post so thoroughly, thank you!

The reason I started looking into this was because of a digital work slip tool http://www.getcrafter.nl/api/) we would like to use. This tool has an API which would allow us to:

• automatically generate work slips and add them to the request,
• synchronise master data,
• synchronise work slip and request status between the two systems,
• automate the closing of requests when the work slips have been marked as completed,
• automatically synchronise the time logging features of Hornbill <> Crafter for reporting

It is standard insofar as that it uses a standard RESTful API with JSON data formatting.

I understand the wish to keep the entire BPM solution as non-technical as possible and agree with you on this. Unfortunately this also means that right now a 'proper' integration with secondary systems is almost impossible to realise without help from Hornbill. 

Is there anything in specific you would need to know about the tool to allow you to evaluate whether or not building an integration would be feasible? An integration like the one in Jira would be terrific for the HB -> Crafter communication. The other way around should be doable with the Hornbill API as described here.
 

Best regards,

Alex

 

[Gerry]

@Alex8000

Thanks for the info, as you say, a typical RESTful API.  The problem here is the need to be able to map values to parameters in and out of the API calls without any programming expertise being required in the BPM.  I understand the requirement, I will have a think about what we could do to help facilitate this, but think this means creating the layer that I mention above, and thats a non-trivial task to undertake. In fact most systems rely on third party tools to provide out-of the-box (or build yourself) integration between cloud applications.  Boomi, SynQ, Muelsoft, Zapier and a bunch of other tools designed specifically to allow integration glue to be developed, packaged and run.  So rather than build our own integration platform (and all the integration connectors for the 100's of apps out there) it might be better to build a connector for Hornbill on one or more of these integration platforms.  Does that make sense?

Gerry

[Alex8000]

Hi @Gerry,

I see, that definitely makes sense. I completely forgot about those API 'middleware' solutions, that would definitely also work! (I'm not that well versed in the whole API <> API thing) 
What would building such a connector for let's say Zapier entail? Facilitating some kind of API triggering/receiving in the BPM (and/or Hornbill in general) would be of immense value to those of us with multiple software solutions, which I think is most if not all of us. @Martyn Houghton would this work for you too?

Best regards,

Alex

[Martyn Houghton]

@Alex8000, @Gerry

Utilising an integration platform would make it easier to manage as long as the integration allows access to underlying entities, i.e. request, so that integration built to the platform can use the entity values in its operation. Like Alex, we would like this to cover both BPM triggered and Hornbill events, however as the latter is currently covered by Web Hooks at the moment our preference would be to focus on BPM triggered events even with limited entity attributes to test the concept.

Cheers

Martyn

[Gerry]

@Martyn Houghton

The way this is normally achieved is the reverse of what you are thinking (I think).  So instead of the BPM having to send all possible variables (for example, all values of an entity) to the service being invoked, what would happen in practice is the integration tool upon receiving an event would make a call back into the Hornbill service to get whatever addtional information it needs in order to complete its work. 

This is true for WebHooks also, when we fire an entity WebHook for a record update, we only send the changed field values as part of the request so its hard to predict which values receive, however, we always send the primary key.   So most uses of WebHooks involve a script that when fired will grab the primary key value its been passed by the invoker and then using an API key (which is generally hard coded into the WebHook script) will call back into the Hornbill service using one or more of the API's to retrieve further information. 

There are a number of reasons its done like this but the main reason is to ensure that all "programming" stays contained within the integration platform only. 

Gerry

[Martyn Houghton]

@Gerry

Thanks for clarification. As long as as part of the integration we can interrogate the entity within Hornbill in order to use within the integration platform, i.e. apply more logic or pass data into the third party system that would be fine.

Cheers

Martyn

[Gerry]

Relevant to this post is this: