710 reads

Why You Shouldn't Use a Database as Your App’s Data Source

by Amir ShevatDecember 1st, 2020

Too Long; Didn't Read

In some use cases, using a spreadsheet as a data source will save you and your users a lot of time and effort. Using the open source integration framework Reshuffle, we use connectors to connect to different services. With so many tools, the more you can consolidate these tools into common tools into one common interface, the easier people can get done with common tools like a spreadsheet. For example, a marketing intern collects user-generated comments on our company’s products, and a tool then analyzes them as Positive or Negative, and generates a report.

Company Mentioned

featured image - Why You Shouldn't Use a Database as Your App’s Data Source

Most apps and APIs use databases as their data source and that is for a great reason. Databases have been designed to be scalable, resilient, and fully featured to support many types of use cases and scenarios. But there is one thing no database was ever designed for: non-technical users. This is why, in most cases, developers need to build an interface over databases, providing non-technical people the ability to add, edit, and delete entries. We do it in every single app we build, calling it the back-office or admin panel. I even worked at companies that called it “God mode.” We build these interfaces and non-technical people hate them just a little less than entering raw SQL, but only just a little. So, what if we use an interface that is already very intuitive to non-technical people? What if they could use a tool they are familiar with—a spreadsheet, for example? Let them be in the comfort of their tool, and interface with that tool as the data source of their workflow or app. If you have little voices screaming in your head right now saying: “But it’s not transactional!” or “It’s not secure!” or “What about data versioning?”—do not panic. I am not suggesting using a spreadsheet-like interface for every use case. I would definitely not run banking software over this architecture. But for some use cases, using a spreadsheet as a data source will save you and your users a lot of time and effort.

BTW, if you hear little voices screaming in your head often, you should probably seek help…

Let’s consider an example of a use case where using a spreadsheet-like interface makes sense. I have picked as my interface—it is a fun and easy-to-use workplace management platform, especially for non-techies.A marketing intern collects user-generated comments on our company’s products. They enter these comments into a spreadsheet after gathering them from Amazon, Shopify, and other retail websites. A tool then analyzes the comments, classifies them as Positive or Negative, and generates a report—because who doesn't love a meaty report?

How to Build It

To connect Monday.com as a data source to my web report and to the sentiment analysis service, I’m using the Reshuffle. It integrates different services, listens to events, and interacts with these services in a standard way. Here is the workflow I want to create:

But wait! Why not call the Monday service directly from the web application? Well, first, that would mean exposing a Monday API token to the web, and that might be a security risk. Second, we want to enrich (and possibly filter) the data that comes from Monday. Doing that in the front end might be a little cumbersome.Let's start by implementing the steps outlined above, fetching the data from Monday, running the sentiment analysis, and serving the data through the API. In Reshuffle, we use connectors to connect to different services, so let's define a Reshuffle App and configure these connectors.

const MY_MONDAY_TOKEN = 'MY_TOKEN'

const app = new Reshuffle()

const nlpConnector = new NlpConnector()
const mondayConnector = new MondayConnector(app, {
 token: MY_MONDAY_TOKEN,
})

Now, we need to define the API endpoint that implements our logic and serves the JSON back to the web application. Reshuffle uses Express under the hood so the semantics might look familiar to you:

const connector = new HttpConnector(app)

connector.on({ method: 'GET', path: '/report-data }, (event, app) => {

// logic placeholder

})

Next, let’s populate the ‘logic placeholder’ by first calling the Monday API to retrieve the comments John collected, and then performing the sentiment analysis:

connector.on({ method: 'GET', path: '/report-data' }, (event, app) => {

const board = await mondayConnector.getBoard(66676)
  
const itemIds = board.boards[0].items.map( ({id}) => Number(id))
  
  const itemDetails = await mondayConnector.getItem(itemIds)

  const data = await Promise.all(itemDetails.items.map( async ({name}) => {
    const nlpResults = await nlpConnector.sentiment(name)
  
    return {text: name, ...nlpResults}
  }))

  event.res.set(headers).status(200).send({data})

})

Lastly, let's initiate the integration by starting the Reshuffle App:

app.start();

Here is what it looks like in real life, starting with the Monday table:

Here is the data returned from the API we created that exposes this information and enriches it with sentiment analysis:

Here is the web report that consumes this API:

Done! No need to build a data entry mechanism or manage the interface for non-technical teams. Reshuffle does the heavy lifting, enabling users to work with the systems they already know versus learning new paradigms or worrying about SQL.Next time you are thinking about using a database and building a back-office interface, consider using the Reshuffle open-source framework and an easy interface like a spreadsheet to implement your workflow.

Now, Make it Happen

As you work with your sales, marketing, and customer experience colleagues, we encourage you to build the integrations that drive better customer experiences and help differentiate your business in the marketplace. With teams using so many different tools, the more you can consolidate these tools into one common interface, the easier people can get work done. Reshuffle is continually listening to what our customers need and desire. Don’t see a Connector to a service you’d like to integrate? Send a tweet to to let us know which Connector you’d like us to develop next.

L O A D I N G
. . . comments & more!