Monolithic vs Microservice architecture with serverless techniques
BITS ZG628T: Dissertation
by
VIPUL AGARWAL
2016HT13368
Dissertation work carried out at
BIZ2CREDIT INFO SERVICES, Noida
[image: image1.png]
BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE
PILANI (RAJASTHAN)
April 2019
Monolithic vs Microservice architecture with serverless techniques
BITS ZG628T: Dissertation
by
VIPUL AGARWAL
2016HT13368
Dissertation work carried out at
BIZ2CREDIT INFO SERVICES, Noida
Submitted in partial fulfillment of M.Tech. Software Systems degree programme
Under the Supervision of
Mr ReetuRaj Banasal ",Senior Project Manager",
IFS Solutions, Noida
[image: image2.png]
BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE
PILANI (RAJASTHAN)
April, 2019
CERTIFICATE
This is to certify that the Dissertation entitled Monolithic vs Microservice architecture with serverless techniques and submitted by Vipul Agarwal having ID-No. 2016HT13368 for the partial fulfillment of the requirements of M.Tech. Software Systems degree of BITS, embodies the bonafide work done by him/her under my supervision.
[image: image3.png]
Birla Institute of Technology & Science, Pilani
Work-Integrated Learning Programmes Division
Second Semester 2018-2019
BITS ZG628T: Dissertation
ABSTRACT
BITS ID No.
: 2016HT13368
NAME OF THE STUDENT
:VIPUL AGARWAL
EMAIL ADDRESS
STUDENT’S EMPLOYING
: BIZ2CREDIT INFO SERVICES, Noida
ORGANIZATION & LOCATION
SUPERVISOR’S NAME
: MR REETURAJ BANSAL
SUPERVISOR’S EMPLOYING
: IFS Solutions, Noida
ORGANIZATION & LOCATION
SUPERVISOR’S EMAIL ADDRESS
DISSERTATION TITLE
: Monolithic vs Microservice architecture with serverless techniques
Traditional monolithic approach with layered architecture has its own advantages. Like simple to develop, test and deploy. Also easy to scale up horizontally (hardware, machines) to run behind load balancer in case of concurrent users increases.
But in real world updates, modifications are inevitable that increases application complexity. As a result the approach does not say much on following challenges
1. For each implementation, entire application must redeploy
2. Change in one unit, whole application has impact/down
3. Parallel development is challenging and complex
4. Dependency on one technology/challenges is in upgrade
5. Reduce fixes/application maintenance/recovery turnaround time.
BROAD ACEDEMIC AREA OF WORKING
: Software Architecture
Key Words
REST: Representational State Transfer
RPC : Remote Procedure Calls
API : Application Programming Interface
[image: image4.png]
Contents
4Chapter 1- INTRODUCTION
41.1
PROBLEM STATEMENT
51.2
SCOPE OF WORK
51.3
PLAN & STATUS OF WORK
6Chapter 2- MODEL: STUDY
62.1
MICROSERVICE ARCHITECTURE
62.1.1
FUNDAMENTALS TO A MICROSERVICE DESIGN: HOW IT WORK
62.1.1.1
PROPERLY SCOPED/BOUNDRIES OF FUNCTIONS
72.1.1.2
PRESENTING AN API STRUCTURE
72.1.1.3
TRAFFIC DATA LOAD MANAGEMENT WITHIN API INTER-COMMUNICATION
72.1.1.4
DATA OFFLOADING
82.1.1.5
MONITERING
82.1.2
ADOPTING MICROSERVICE ARCHITECTURE STRAINGHTAWAY BENIFITS
92.1.3
SERVERLESS CONCEPT
10Chapter 3- MICROSERVICE ARCHITECTURE ADOPTATION
103.1
APPLICATION STUDY: CUSTOMER On-BOARDING/ REGISTRATION
103.1.1
OBJECTIVE
103.1.2
FUNCTIONAL REQUIREMENT SPECIFICATION: PROCESS MAPPING
103.1.2.1
REQUIREMENT MAPPING
103.1.2.2
IN-SCOPE FUNCTION (PROTOTYPE)
103.1.2.2.1
USER CREATION/SIGN UP
103.1.2.2.2
SCORE GENERATION
103.1.2.2.3
LOGIN SCREEN
113.1.2.2.4
CUSTOMER EXISTING CHECK AKA DEDUPE PROCESS
113.1.2.2.5
DASHBOARD IMAGE/DOCUMENT UPLOAD
123.1.2.3
USE CASE [FULLY DRESSED]
123.1.2.3.1
User Creation / Sign up
133.1.2.3.2
Score generation
143.1.2.3.3
Login
153.1.2.3.4
Existing customer check aka dedupe process
163.1.2.3.5
Image/Document upload option screen
173.1.2.4
ER-DIAGRAM [GROUPED]
18Chapter 5- Architecture of Microservices
185.1
Deployment Architectures of Services
19Chapter 6- Application: Pseudo and Folder Structure
226.1
Pseudo code of Sign Up service
236.2
Pseudo code of dedupe process service
236.3
Pseudo code of Getscore service
246.4
Pseudo code of Login service
246.5
Pseudo code of Image Document Upload service
25Chapter 4- ACKNOWLEDGEMENT
26Chapter 5- CONCLUSION SUMMARY
27Chapter 6- References
28Checklist of items for the Final Dissertation Report
Chapter 1- INTRODUCTION
Traditional monolithic approach with layered architecture has its own advantages. Like simple to develop, test and deploy. Also easy to scale up horizontally (hardware, machines) to run behind load balancer in case of concurrent users increases.
But in real world updates, modifications in the application model are inevitable that increases application complexity. Also many a times multiple implementations are require by marketing team as business requirements. As a result monolithic approach does not say much on following challenges
1. For each implementation, entire application must be redeploy
2. Change in one unit, whole application has impact/down
3. Parallel development is challenging and complex
4. Dependency on one technology/challenges is in upgrade
5. Reduce fixes/application maintenance/recovery turnaround time.
1.1 PROBLEM STATEMENT
· For each implementation, entire application must redeploy
· Change in one unit, whole application has impact/down
· Parallel development is challenging and complex
· Dependency on one technology/challenges is in upgrade
· Reduce fixes/application maintenance/recovery turnaround time.
· Architectural approach that consider/include following features
1. Component Re-usability
2. Adopt changes without impacting the whole application
3. Faster development
4. Technology freedom
5. Reduce fixes/application maintenance/recovery turnaround time
· The objective is to divide whole application into smaller services that can communicate to each other, instead of making one application. Each microservice is a small application that has its own business logic along with various adapters. Inter Communication can be by expose by a REST, RPC or message-based API
· The Microservice architecture pattern has significant impacts on the application and the database relation. Instead of using one database, each microservice has its own database. To have database per microservice is essential to benefit loose coupling property. That allows adopting changes without impact on whole application, parallel development, technical freedom and upgrade and reducing TAT for downtime repairs.
· Each microservice can be pluggable unit, for the same business need in any application
1.2 SCOPE OF WORK
· Studying microservice architecture with serverless techniques and concepts to implement for application.
· Application for User registration, Login, Socio creditability score, business search, customer document Image file upload for tagging functions
Analyzing the performance, consumed efforts and long-term benefits in setting up
1.3 PLAN & STATUS OF WORK
S.No.
Description of Work
Start Date
End Date
Status
1.
System Study:
· Requirement analysis
i. By adopting microservice architecture following straightaway benefits can be taken
· Study microservice architecture patterns
i. Microservices Architecture
ii. Fundamentals to a successful microservices design: How it works
· Serverless techniques
i. What is serverless
· Conclusion
January 21, 2019
January 26, 2019
Done
2.
Info pre-processing and mapping:
· Information collection
i. Info pre-processing and mapping: FRS
ii. Use Cases [Fully dressed]
· Transforming of information
i. Grouped ER-Diagram
· Information cleaning and massaging
January 27, 2019
February 22, 2019
Done
3.
· Design architecture
· Identifying microservices to server scope of work
February 16, 2019
March 8, 2019
Done
4.
· Input output parameters and request response
· logging and analysis Inter microservices communication channel (RPC/REST)
March 9, 2019
March 25, 2019
Done
5.
Report documentation/submission
March 25, 2019
April 1, 2019
Done
Chapter 2- MODEL: STUDY
2.1 MICROSERVICE ARCHITECTURE
The foundation of microservices architecture (MSA) is about developing a single application as a suite of small and independent services that are running in its own process, developed and deployed independently.
Microservices follow the principles of service-oriented architecture (SOA) design. SOA Principles of Service Design are as follows:
· Standardized service contract (services follow a standardized description)
· Loose coupling (minimal dependencies)
· Service abstraction (services hide their internal logic)
· Service reusability (service structure is planned according to the DRY principle)
· Service autonomy (services internally control their own logic)
· Service statelessness (services don’t persist state from former requests)
Although SOA and microservices follow similar principles SOA has an enterprise scope while microservices have an application scope. Within the enterprise scope, apps communicate with each other.
2.1.1 FUNDAMENTALS TO A MICROSERVICE DESIGN: HOW IT WORK
2.1.1.1 PROPERLY SCOPED/BOUNDRIES OF FUNCTIONS
Issue with monolithic architectures is there is so much code that needs to implement. And to make any change to a monolithic app, you must coordinate across different development teams in order to ensure that everyone’s code continues operating properly. This is time consuming.
For this reason, boundaries of functionality should be wisely defined for implementing. So on one hand there should not be overpartition their functionality that ended up with too many small microservices. And on the other hand need to avoid too much to stuff into each service.
One way to define the proper scope is to partition the services along logical functionality lines. For example, if you have a credit score (for a typical money landing application) calculation logic function in your monolithic app that many other functions call, it is a clear cut candidate to be broken out into its own service.
Another scoping approach is to mirror the development company’s structure. Each application subgroup (e.g., the Login authentication group responsible for user identity and authorization) takes responsibility for creating one or more microservices for the functionality that falls into its area.
2.1.1.2 PRESENTING AN API STRUCTURE
Once boundaries into multiple co-operating microservices of a single application defined, the obvious bigger question arises that is how the services should communicate to each other. Traditionally, this is REST web services API calls(Or can use other transport mechanisms).
Presenting an API to calling services in some way represents the old challenge of integration. For an overall application to run properly, each of the individual services must be able to reliably communicate, send and receive data. Testing the APIs operate properly is necessary to ensure that everything works together.
Instead of jumping into API coding immediately, do some homework on paper to define what a specific service must expose to operate properly. It may take little iteration to fully flesh out an API capable of presenting the service behind the API, as well as managing the calls from multiple client types.
2.1.1.3 TRAFFIC DATA LOAD MANAGEMENT WITHIN API INTER-COMMUNICATION
Once the API structure is up and one service can communicate other, need to brainstorm exception handling and worst case scenarios.
In the real world of operations applications, a service may run slowly, take a long time. Or a service might stop running/responding due to a software or hardware crash. And sometimes a too many request calls for the lower-level service to respond quickly enough.
Addressing this heavy traffic situation requires management. There must be a way for calling and called services to communicate status and coordinate traffic loads. This is commonly referred to as a “circuit breaker pattern.”
From the perspective of the calling service, it should always track its calls and be prepared to terminate them if the response takes too long. From the perspective of the called service, the API design should include the ability to send a response that indicates overload. This response, typically referred to as backpressure, signals that the calling service should reduce or redirect its load.
2.1.1.4 DATA OFFLOADING
The erratic traffic of microservice applications mean that individual services come and go. Virtual machines crash, fail to respond, while individual services instances are transient, the overall service must be available and continue operating so that users will keep getting results.
To ensure that users can continue to perform useful work when one instance from which their sessions are being served fails, you can migrate user-specific data off of service instances and into a shared, redundant storage system that's accessible from all service instances. In this way, you can ensure that no instance crash stops user interactions.
A further twist on the offloaded storage approach is to insert a shared, memory-based cache system between a given service and the storage associated with that service. This allows for quicker data access and improves application performance. Naturally, the caching system becomes another service in the application architecture, and makes the overall application more complex, but data offloading and caching improves application performance.
2.1.1.5 MONITERING
Decomposition of a monolithic application, along with insertion of offloaded data layer and data caching to increase performance, inevitably means a more complex application topology — a lot more complex.
The monitoring system for a microservices-based application must allow for ongoing resource change, be able to capture monitoring data in a central location, and display information that reflects the frequently changing nature of microservices applications.
Trying to manually correlate errors across a service cascade is nearly impossible, so use a monitoring system that can discover and display events based on a common timeline to support root cause analysis.
Most microservices monitoring systems place a monitoring agent on each service instance, where it can track specific instance data. These monitoring systems can also capture application-created log information. All of this data migrates to a centralized database, where the system does cross-correlation, allowing monitoring alerts or humans to track important event data.
2.1.2 ADOPTING MICROSERVICE ARCHITECTURE STRAINGHTAWAY BENIFITS
By adopting microservice architecture following straightaway benefits can be taken
i. Improved scalable application. Because microservices let you independently scale services up or down, the ease—and cost—of scaling is dramatically less than in a monolithic system. Adding new capabilities usually means adding discrete new microservices, not redoing the entire application, which increases both development speed and application stability.
ii. Better fault isolation/tolerance. If one microservice is in trouble, all the others will likely continue to work. This is a key part of the microservices architectural design.
iii. Localized complexity/dividing complexity of large application. Microservice architectures allow developer think about services as black boxes. Owners of the service need to understand the complexity of only what is within their service. Other service owners need to know only what capabilities a service provides, without having to worry about how it works internally. This compartmentalization of knowledge and complexity makes it easier to create and manage large applications.
iv. Increased business agility/Risk taking to faster implementation of business. Microservices are relatively small and simple, and failure of a microservice affects only that service—not the whole application—so enterprises can afford to experiment with new processes, algorithms, and business logic. Microservices give you the freedom to experiment and “fail fast.”
v. Increased developer productivity. Now developer can get up to keep velocity rapidly, since it’s easier to understand a small, isolated piece of functionality than an entire monolithic application.
vi. Simplified debugging and maintenance. For the same reasons that building individual microservices is easier than coding for a monolithic architecture, developers can be much more productive when debugging code and performing maintenance.
vii. Better alignment of developers with business users. Since microservice architectures are organized around business capabilities, developers can more easily understand the user perspective and create microservices that are better aligned with the business.
viii. Future-proofed applications. When innovations happen and new or updated technology disrupt your software development process, microservice architectures makes it easier to respond by replacing or upgrading the individual services affected without impacting the whole application.
2.1.3 SERVERLESS CONCEPT
Serverless is an approach that objective is to address infrastructure and software architecture issues. Infrastructure is expensive to run. Adopting microservices and running more servers with multiple environments is expensive. Servers might be busy handling requests or sitting idle, but the cost to the developer remains the same.
The term 'serverless' is somewhat misleading, as there are still servers providing these backend services, but all of the server space and infrastructure concerns are handled by the vendor. Serverless means that the developers can do their work without having to worry about servers at all.
For clarity, “serverless” means the developer does not have to think about servers, even though they exist. AWS handles them. Plus, Lambda only charges you when your code is run, which is an efficient pricing model.
· Using a managed compute service such as AWS Lambda, Azure Functions, Google Cloud Functions, IBM Open Whisk, or Auth0 Web Task to execute code
· Leveraging third-party services and APIs in a more thoughtful way
· Applying serverless architectures and patterns.
Chapter 3- MICROSERVICE ARCHITECTURE ADOPTATION
3.1 APPLICATION STUDY: CUSTOMER On-BOARDING/ REGISTRATION
3.1.1 OBJECTIVE
The objective is to write this page, is to gather complete requirement of business side first. Note down each activity details of business and make all the use cases with full details by considering exceptional situations as well.
3.1.2 FUNCTIONAL REQUIREMENT SPECIFICATION: PROCESS MAPPING
3.1.2.1 REQUIREMENT MAPPING
Background: The ultimate objective is to deep study of microservices architecture over the traditional monolithic approach. Also try to do all feasibility study and adaptation method, which leads the journey from monolithic to microservices architecture that must cater various business requirements of implementations.
Hence the prototype model must of generic which should cover up various requirements of business scenarios.
3.1.2.2 IN-SCOPE FUNCTION (PROTOTYPE)
Customer engagement work flow. Where a potential customer will be onboard and system responsibility is to create a ‘Lead’ basis on basic customer details and validating his data. This is generic scenario of any customer engagement platform for various implementations like BFSI, money landing, insurance sector etc to avail their services.
Following are high level activity function points:
3.1.2.2.1 USER CREATION/SIGN UP
Customer landed up to first page i.e. User creation /sign up page. He will ask to fill his basic details like last name, First name/ mobile number with OTP verification / PAN / aadhar/ email/ dob/ Annual salary range / occupation/ no of dependent/sex/married/ address city.
3.1.2.2.2 SCORE GENERATION
Configurable rule policy engine(): Basis on above details his for Lead generation eligibility check system executes score card generation on configured rule policy for the customer. If the generated score in the acceptable allowed range limits a Lead will be will be created of customer and welcome mail send to his registered email id with customer password. Email id will be of his userid for login purpose.
3.1.2.2.3 LOGIN SCREEN
Screen option will be provide left corner of top of screen for him to get login into the system once registered for move to next stage. System will be validate his user password credentials as per system generate security management policy configuration.
Also option for the change password, authenticate password / forget password to set up new one as per customer wish and system will be allow him to set password as per the configured password policy.
3.1.2.2.4 CUSTOMER EXISTING CHECK AKA DEDUPE PROCESS
system will execute dedupe check for the customer entered details and find out if the customer is existing or not basis on the input data parameters and identify his CIF number. If customer is new then new customer id i.e. New CIF number will be generate.
3.1.2.2.5 DASHBOARD IMAGE/DOCUMENT UPLOAD
Once lead creates or existing customer successfully login, system takes his journey to the next stage and landed onto customer dashboard. System further ask customer to upload his image/documents.
Here system has added provision to identify / Tag the uploaded images and check whether the uploaded image is, his face/ his house image/ or his Identity document image. An advance program will be of image tag feature and data processing will execute to tag the image. The program executes on the probability percentage of image tagging if in case wrong image has been uploaded a text message of wrong image would be send to the customer asking to check uploaded image and upload again with the corrected one.
3.1.2.3 USE CASE [FULLY DRESSED]
3.1.2.3.1 User Creation / Sign up
Title:
Register/Sign Up for application
Description:
User accesses the available system for him to register.
Primary Actor:
End User customer
Preconditions:
User is sign Up into system by filling up all basic details
Postconditions:
User gets Registered and will receive welcome mail with crdentials
Main
User able to sign up
Success Scenario:
User Fill basic details
User is new a first time visit
User clicks “Submit” button.
System registers user and displays a confirmation message.
Extensions:
User already Exists
User Provided wrong information or mail id does not exists
User refuse to accept T&C Links
Use does not fill mandatory fields Items
Mobile number is wrong
3.1.2.3.2 Score generation
Title:
Score Generation
Description:
Score matrix has been configured e.g. Age of Range > 25 ~35 < gets maximum else medium. However No score for 60 age. Similarly professions like IT professionals, service class engineers’ gets maximum score however professions like politicians, lawyer have no score. Finally cumulative score of each rule between range of >300~400< is Ok and
