Overview

NoteWeb offers packages of frequently used functionality that you can combine to handle your most typical cases in a standard that you define effortlessly, while taking notes. Herein "effortlessly" means tagging your content in UI. List of frequently used functionality supported by NoteWeb includes simple things like forms, file attachments, diagrams, chats, dashboards, scientific notebooks, but also more specific gears like Apache Spark workloads, business rules and processes, cloud accounts.

As often the case with effortless changes and automation, a week or more of active NoteWeb use might raise a question of governance, once you start analysing how and what content is created. The latter can be approached from different perspectives. Bottom-up implies audits, alerts, metrics monitoring, logging, etc. Top-bottom means some kind of blueprint for each solution, that solution components adhere to at design- and at run-time.

NoteWeb offered tools to define such blueprints from its first versions years ago - diagrams, mindmaps and so on. Though NoteWeb's DL4j personalized network started suggesting tags for your updated content in 2017Q2, it still does not jump above average industry level on NLU and image recognition tasks. While tools like GPT-3 or OPT-175B can already cause different illusions also with tech savvy folks, NoteWeb still strives to offer business value, when you unleash NLP onto your content and configurations stored therein.

This situation means, in plain words, that whatever you use to define your blueprint - diagrams, plain text or ad-hoc content models, it is still up to another human or yourself to translate the design into programming artifacts and validate those artifacts for purpose suitability. That's unless you know and follow a set of rules - some strict, some laxer for your blueprint, like UML (or any other model-driven engineering notation), higher level programming language syntax or a DSL, to make the blueprint. This article is about how NoteWeb supports the latter approach, once again, while you take your notes.

Turning Notes to Blueprints

Most points from the previous section are well known. So while NLU muscle builds up, NoteWeb suggests to make blueprints out of your plain text notes. Those of you with experience of a Solution Delivery Manager, an Architect, Developer, DevOps or QA, might remember colleagues who constantly type something on requirements elicitation sessions. I'm such a guy. Here is how to start a journey to a system solution blueprint, yet while gathering the requirements for it. The idea is very simple:

  • type resolution on each point you hear as fast as you can
  • reflect context nesting with new line indentation
  • adopt simple referencing/acronyms convention
  • capture entities, attributes, resources, assets, relationships, rules, operations, dates and numbers
  • dedicate at least a line to each statement, specify less important/not yet known attributes after more important/known ones, use new lines to separate blocks
  • review your notes fixing references and indentation right after the call or as soon as you find a few minutes for it

For each entity that represents a cloud resource, a service, etc prepend keywords identifying resource type:

  • context - groups resourcess and assets belonging to the same solution (NoteWeb context)
    • cloud - groups resources and assets of the same cloud account
      • bucket - bucket for object storage (AWS S3 buckets, GCP bucket, Azure blob storage container)
      • vpc - virtual private cloud network hosting nested network resources and assets (AWS VPC, GCP VPC, Azure Virtual Network)
        • vm - virtual machine instance (AWS EC2 VM, GCP Compute Engine VM, Azure VM)
        • load balancer - cloud load balancer to route traffic to multiple endpoints (AWS ELB, GCP LB, Azure LB)
      • resource - a pre-existing resource like RDS MS SQL database (is not to be provisioned at runtime)
        • relational table
        • document table
      • user
      • container
      • kubernetes service
      • app service
      • lambda
      • queue
  • switch
  • read input
  • call service with into
  • operation published
  • ... a few other control flow primitives

Code Snapshot

This article refers to earlier version of noteweb-paas-dsl-service with an interim code snapshot developed by the author and published just for this article at:

. Subsequent blog posts from this series would cover developing a backend supporting deployment of a solution per blueprint in the web note code sample below.

 

context "NoteWeb"
  cloud "NoteWebInAWS" type "AWS"
    resource "notesDatabase" type "mysql" url "..."
      relational table "notes" from "notesDatabase"
      relational table "tags" from "notesDatabase"
    resource "auditDatabase" type "dynamodb" url "..."    
      document table "audit" from "auditDatabase"
    bucket "attachments"
    user "NoteWebSystem"
    vpc "NoteWeb"
      vm "NoteWebSearch" type "t2.micro.search" image "arn:...::"
    container "NoteWebPython" image "arn:...::"
    kubernetes service "NoteWebWildfly" image "arn:...::"
    app service "NoteWebNode" image "arn:...::" port 80
    lambda "DirectAttachmentUpload" image "arn:...::"
    queue "audit" type "sqs"
    queue "events" type "sns"

operation "init"
  read input tenant
  switch:
    tenant == "" ->
      call service NoteWebWildfly "configChecks()" into "instanceConfig"
      call service NoteWebWildfly "notifyNoteWebUser($instanceConfig)"
      exit
    otherwise ->
      read input language, country, admin_email
      call service NoteWebWildfly "setupDefaultTenant($tenant, $country, $language, $admin_email)"
      call service NoteWebWildfly "configChecks()" into "instanceConfig"
      switch:
        admin_email == "" ->
          call service NoteWebWildfly "notifyNoteWebUser('New NoteWeb installation's default tenant: ', $instanceConfig)"
          exit
        otherwise ->
          call service NoteWebWildfly "sendEmail($admin_email, 'New NoteWeb tenant setup: ', $instanceConfig)"
          exit

This blueprint is in the DSL and conforms to approach outlined above. Lines highlighted in bold correspond to the code snapshot accompanying this article. This blueprint defines about a tenth of containers and resources used by NoteWeb deployment at http://noteinweb.com and is purely illustrative.

This first code snapshot introduces:

  • Scala parser combinators library to develop lexer and parser for the NoteWeb PaaS DSL
  • a notion of AST - Abstract Syntax Tree and a resource - an abstract resource or asset to be provisioned or existing in a cloud
  • CloudTransformer - performs control plane cloud provider API invocations based on supplied resource objects
  • CloudActor - invokes data plane cloud provider API calls (to be used in later integration scenarios)
  • CloudInspector - services other system components with information about existing cloud resources
  • provides examples of building a CDKtf construct and using Java AWS SDK in Scala

Bakus-Naur Form for NoteWeb PaaS DSL

Code snapshot provided supports a grammar approximately defined in the following BNF-snippet.

<block> ::= (<contextStatement> | <statement>)+

<contextStatement> ::= "cloud" INDENT (cloudStatement)+ 
<cloudStatement> ::= "user" 
              | "bucket" <string literal> 
              | "vpc" <stringLiteral> INDENT (<vpcContent>)+ DEDENT
              | "resource" <string literal> "type" <string literal> "url" <string literal> INDENT (<resourceContent>)+ DEDENT
              | "container" <string literal> "image" <string literal>
              | "kubernetes service" <string literal> "image" <string literal>
              | "app service" <string literal> "image" <string literal>
              | "lambda" <string literal> "image" <string literal>
              | "queue" <string literal>
<vpcContent> ::= "vm" <string literal> "type" <string literal> "image" <string literal>
              | "load balancer" <string literal>
<resourceContent> ::= "relational table" <string literal>
              | "document table" <string literal>


<statement> ::= "exit"
              | "read input" (<identifier> ",")* <identifier>
              | "call service" <identifier> "with" <stringLiteral> ("into" <identifier>|<string literal>)
              | "switch" ":" INDENT (<ifThen>)+ [otherwiseThen] DEDENT

<ifThen> ::= <condition> "->" INDENT <block> DEDENT

<otherwiseThen> ::= "otherwise" "->" INDENT <block> DEDENT

<condition> ::= <identifier> "==" <stringLiteral>

 

Series Scope

Subsequent articles provide later code base snapshots and walk through the following topics:

  • authentication against cloud provider accounts as defined in "cloud" statements
  • invoking different types of services as defined in "call service" DSL statements
  • deploying Kubernetes services with Kubernetes Operator SDK per blueprint defined in "kubernetes service" statement

There is more material to cover, but agenda'd follow the interest from the market.

 

References

  1. Towards a Lightweight Multi-Cloud DSL for Elastic and Transferable
    Cloud-native Applications https://www.scitepress.org/papers/2018/66838/66838.pdf
  2. Building a lexer and parser with Scala's Parser Combinators by Pedro Palma Ramos https://enear.github.io/2016/03/31/parser-combinators/
  3. Programming in Scala, Third Edition by Martin Odersky et al.

 




Note In Web, Inc. © September 2022-2024; Denys Havrylov Ⓒ 2018-August 2022