Software Semantic Evolution: Part 2

By on

Learn more about Yefim (Jeff) Zhuk. Click here to read Part 1Part 3, and Part 4 of this series.

Microservices, RAML, DataSense by MuleSoft, and the Next Step

RESTful API Modeling Language (RAML)

Have you worked for an enterprise that developed thousands of services? At some point you might have noticed that it is easier to create another one than find an existing service that covers the needed function. This very sad discovery is a good indication that service discovery needs improvements.

RESTful API Modeling Language (RAML) is designed to provide these improvements. RAML offers developers a formal way of describing RESTful APIs.

What is RAML?

RAML is built on the top of the standards such as YAML and JSON. RAML is a non-proprietary, vendor-neutral open spec. RAML gives developers freedom of providing their own semantics to define specific properties of services in a specific business domain. At the same time RAML includes basic characteristics necessary to invoke a service, such as basicUri (usually serves as the endpoint of REST service invocation), describes the post and get queries, and queryParameters that must be provided with the RESTful call.


JYZ Example

This example provides human readable descriptions as well as formal method definitions.

To get a well-formatted response use the PDF type with a simple request:

Note, that it is up to a developer to choose specific semantics for data property names, such as courseType, courseTitle, and courseInstructor. These naming conventions that look obvious and even trivial for one group of developers might miss expectations of another company, which has a different business dialect. And we will talk about semantic integration a bit later.

Mule Soft Introduced DataSense

Mule Soft made another step in this direction by creating Data Sense metadata for application designers.

MuleSoft is actively moving to Microservices. For developers this move to Microservices means API-led development. This is exactly what MuleSoft offers.

Similar to the discussion on service layers we had before, MuleSoft also separates service layers into three categories: Experience, Process, and System Layers.

The lowest service layer called System Layer represents underlying utilities and data services, including APIs to applications that provide data. The Process Layer is responsible for business processes that form workflows and eventually integrated into the Experience Layer consumed by end users.

MuleSoft Layers

DataSense is a better tool for developers who usually described their design ideas in Power Point and diagrams. Data Sense allows creating metadata to facilitate application design. Anypoint Studio can understand these metadata and can provide necessary translation data type and structure described there into application body.

At this point Anypoint Studio does some work on behave of developers. The tool intelligently discovers information about internal and external resources. Usually this was manually done by people. Imagine that there is a mobile application connected to Facebook. Facebook has its own API, data types and structure, which can be captured by DataSense. Anypoint Studio can provide this information back to you helping you to make better and quicker decisions about interfacing Facebook.

What Can be Done with DataSense and Anypoint Studio?

In the terms of Anypoint Studio, two major functions to discover and describe metadata are Perceptive Flow Design and DataSense Explorer.

Perceptive Flow Design

Mule can use an existing connection to the resource to retrieve metadata about message properties and payload.

This information will feed into DataWeave, a message transformation component. Then, the mapping data from one format to another happens almost automatically. At least part of work is done by a computer!

DataWeave Template Engine

You can type the word payload in the Anypoint Studio GUI to get a list of all the properties and methods associated to the payload. This is close to magic!

DataSense Explorer    

 DataSense Explorer

DataSense Explorer is part of Anypoint Studio. Explorer can visualize the message data structure at different points of the flow, when a developer is still designing the flow. A developer can select any element in the flow and the DataSense Explorer will display the structure of input and output data.

With the DataSense Explorer a developer can see the message contents at any given point in the flow. This is possible because the Explorer has access to the DataSense metadata of compatible connectors and knows about Session Variables, Inbound and Outbound and Payload properties.


DataSense and Studio Connection

DataSense allows developers to describe and discover information via the connector and connection to the application. Then DataSense passes this information about application entities and their structures to Anypoint Studio. Anypoint Studio presents the data at design time. Studio can even make suggestions about the expected values in fields returned by the connector. These suggestions are based on connector’s metadata and DataWeave’s intelligence.

What does it mean to implement DataSense?

The implementation consists of the following:  configuring metadata retrieval by creating the connector to supply this information, and configuring metadata awareness with annotations of operations (methods), providing to Anypoint Studio necessary information about the DataSense implementation.

Anypoint Studio is a rich extension of Eclipse with mostly well-known and almost intuitive windows. You can open the Import wizard from the File menu. With a pop-up wizard you can select an existing Anypoint Studio Project from External Location or open a new one. Once you select the Server Runtime as Mule Server 3.6.0 CE or EE, the studio will display the Mule Flows.

For precise settings you can use the file with access credentials and more data describing the connector.

Connectors with Static and Dynamic Data Models

A connector might have a Static “strongly typed” data model or a Dynamic Data Model where data types are resolved at run-time

In the case of Static model, metadata retrieval as well as metadata awareness is immediately available by the strongly typed parameters.

In the case of Dynamic Data Model, some metadata will be resolved at run-time with two annotated methods, getMetadataKeys() followed by getMetadata().

The image below from MuleSoft examples shows one of the GUI windows offered by Studio for configuration.


The bottom line: DataSense and Studio are working together to discover and describe application interfaces. This work saves developer’s time and improves precision of the design by bringing an important layer of metadata information at design time.

The Trend

Do you see the trend? Development and even its modeling part become increasingly formal. Abstract ideas find their precise expressions and become tangible. Computer programs can understand and test these ideas. In a similar way the abstract concept of interfaces became part of Java language to allow compiler checking for correct implementations.

We describe software program behavior and support this ideology with Behavior-Driven Development tools, such as Apache Cucumber.

And we try to formalize design with simple language, so business people can understand and participate in the design. We did not want to introduce another weird xml languages (like RDF family). This intention is clear. Streamline the development flow and give the business more opportunities to directly participate. This is coming, but coming extremely slow. Here is just one of the reasons.

New tweaks of technology often require to restore old development arts in completely new environments. For example, application frameworks, such as Hibernate and others, hide from us complexity of data and degraded the need for data modeling skills.

Big Data renew this demand. When millions and billions of records are at stake, perfect data modeling is required. Perfect modeling with perfect understanding of NoSQL DB features and correct expectations of application queries.

There are more reasons telling us that it would be a stretch to expect business masses joining us in development trenches next week or next year.

So, what is the next step of software evolution?

Leave a Reply