visit
According to the paper published by Lokman Rahmani et al., the S/Kademlia distributed hash table (DHT) used by the ACN is resilient against malicious attacks.
What is mysqldump?
PostgreSQL is the jack of all trades when it comes to databases. It brings you all the features you've come to love about SQL plus a bunch of features from Non-SQL databases. Some of these Non-SQL features like the JSONB data type are wonderful and you don't even have to dare flirt with another database. Others are good but not as robust and featureful as other databases.
Recently I needed to delete some documents that I saved in MongoDB after some time. I can think of a few examples of why we would want to delete data after some time:
Connecting a database to a Java application is not an easy process. You need to consider the connection pool, the data access layer, etc.
Metabase is a business intelligence tool that lets you access your data in a read-only manner.
Investing in customer data is a top priority for marketing leaders.
First of all what is Stored Procedure?
Learn what tools and approaches are available to create and load test data for webapps without an ORM.
For now, I am assuming that you all know what is SQL or you have some experience with SQL or you are using SQL for many years. I know many people who know so much about SQL and even have created many projects based on SQL and they do not know what is SQL engine or how actually SQL database engine works. It’s obvious no one’s care about the internal working of SQL engine or something because without knowing How SQL works we can still create and access database using any SQL program.
Having worked on a project for 6 months using GraphQL on the backend, I weigh up the technology’s fit into the development workflow
Image a situation wherein you have just launched your app. But the data of your app is not being properly displayed or you are not able to fetch the data that is being entered by the users. What will be the impression of your app in the user’s mind?
“A man stretching out his two hands with careless written on them in black and white” by Mitchel Lensink on Unsplash
There are several architectures that could be followed to achieve multi-tenancy at the database layer.
Recently I have introduced Directual on Hackernoon (see Low-code for hackers). Now I would like to continue with absolutely practical post. First, I will show the general scheme of combining Directual and React, and then you will find the live-demo of building an app from scratch to production-ready Docker-container.
Race conditions in a database and how you can solve them using techniques such as pessimistic and optimistic concurrency control.
Explaining the Database Plus concept, its impact on system architecture design, and its innovations: from Proxyless Service Mesh to microservice backend support
As of July 13th, 2022, there are 135 security flaws reported to the CVE database. Here are 8 essential measures you can take to protect your PostgreSQL server.
A Zero to SQL 90 Days Roadmap
In this article, we'll deep dive into all the basics to help you hit the ground running when it comes to AWS RDS.
One of these issues you might stumble across using the PHP framework Laravel is this one:
The number of security breaches and cybercrimes is increasing rapidly. With more and more approaches being transferred online, hackers have found their way of hacking into a system and corrupting the information or stealing data to turn it into profits. As the technology keeps on changing, the hacking attempts are also becoming smart and upgraded to ensure hackers are never caught in action.
“Dashboard in the pilot's cockpit” by Mitchel Boot on Unsplash
MongoDB is a document-oriented NoSQL database for storing large amounts of data. MongoDB uses collections and documents instead of tables and rows
This brief article on VIews in SQL explains how to create a view and other various operations in a step-by-step explanation.
ChatGPT is truly impressive. You can instruct it to do all sorts of things when they can be communicated in plain text.
Get started with MariaDB with just three steps using the official MariaDB docker container.
Why do we need Edge Computing? Why is the cloud not the solution for everything? Why is Edge Computing adoption taking so long? What's the solution to it?
An interview with the CEO of Berlin-based Startup solving local on-device data storage and decentralized data-flows out-of-the-box to empower Edge Computing.
For many businesses the lack of data isn’t an issue. Actually, it’s the contrary, there’s usually too much data accessible to make an obvious decision. With that much data to sort, you need additional information from your data.
Connectivity is something amazing. Right now, we are used to use our computers or phones to buy, post, watch, etc. We can do lots of things actually. We are connected to the world and to each other.
DuckDB is doing for embedded analytic workloads what SQLite did for embedded OLTP workloads.
Have that old laptop that's just in the back of your closet? Figure out how to give it a new life!
We get a glimpse into the inner workings of a valuable company and it turns out it's not all sunshine and rainbows.
A data lake is totally different from a data warehouse in terms of structure and function. Here is a truly quick explanation of "Data Lake vs Data Warehouse".
Explore how exactly distributed storage works in Hadoop? We have to characterize an essential node (known as NameNode) from one of the workers (DataNodes).
With the launch of Rails 6, one of the new features that were announced was of multiple database support.
Photo by Madison Grooms on Unsplash
Most apps and APIs use databases as their data source and that is for a great reason. Databases have been designed to be scalable, resilient, and fully featured to support many types of use cases and scenarios.
Let’s say we have to query a user table with a metadata JSONB column on a PostgreSQL 9.5+ database.
I spent the better part of a day moving a project completely off of the ORM we were using (TypeORM). Ever since I started doing web development, I’ve been using some kind of heavy abstraction on top of my database. Whether it’s Mongoose when I used MongoDB, or TypeORM when I decided to move to TypeScript and PostgreSQL. However, it turns out that ORMs and query builders aren’t so great. I used to be that person that thought ORMs and query builders did the job for me, and I used it as an excuse to not learn SQL (which is what everyone tells you NOT to do). Given the tools that exist today, I would wholeheartedly recommend doing it the opposite of how I did if you’re starting off new to these things — write your SQL stuff by hand, or with help from the tools I mention later in the post. It’ll feel really slow at first, but you’ll thank yourself later 😄
Organizations nowadays are storing and performing computation of the data on the cloud instead of handling themselves. Cloud Service Providers (CSPs) provide these services at an affordable cost and low maintenance. But to ensure compliance and retain privacy, organizations need to transfer the data in an encrypted format, which does ensure the confidentiality of the data. However, once the data reaches the cloud, the CSP has to decrypt the data to perform operation or computation.
Normal development flow requires continuous patching the production database with local changes normally made automatically by the orm software, this method is not perect but deceptively simple, all we'll use is standard Unix commands, and is good enough for us.
Once upon a time, a company I worked for had a problem: We had thousands of messages flowing through our data pipeline each second, and we want to be able to send email and SMS alerts to ours users when messages matching specific criteria were seen.
A comparison of data models used in open-source SQL and NoSQL time-series databases for IoT.
In this article I want to share my experience of how to setup MySQL replication locally with using Docker.
This guide explains how to use React and Redux to fetch data from JSON files in an easy and lightweight method through a real example and well explained steps.
An extraordinary computer science breakthrough called Accord is bringing globally available, general-purpose ACID transactions to the next Cassandra release
Guide on How to do CRUD operations like DynamoDB GetItem, PutItem, DeleteItem and UpdateItem using AWS Serverless and NodeJS.
In this post, we highlight some key differences between a Customer Data Platform (CDP) and other tools generally used in a marketing tech stack. We also tackle the all-important question on many companies’ minds: “should I build or buy a CDP?.”
GraphQL is awesome! But one of the most annoying parts to implement a GraphQL server is solving N+1 problem.
Learn Why And How To Use Relational Database Migrations. The way to initialize and update schema for relational databases with Java.
Many developers still install a database on their computer and work on it. It may be a preferable method, but using Docker takes only seconds to set up a database. I used PostgreSQL and Docker in my latest project. As a result, I have achieved efficiency and convenience. I will give you my experience with PostgreSQL and Docker.
Since being forked from MySQL back in 2009, MariaDB has become one of the most popular databases of choice for developers over the past decade. While many technologists have likely gravitated to it as a solution due to its open source roots and that it's rooted in the relational database world, that really only begins to scratch the surface of what MariaDB has to offer.
Big data is a big problem, at least getting anything useful out of it. Every day there is about three quintillion (the next step up is sextillion or one zettabyte) bytes of data created and only about 20% of it is structured and available to easily process. Nearly all useful processing that is done relies on a philosophy that is little changed from the green bar reports we were generating during the night shift and handing out up till the turn of the century. The whole map/reduce process is overnight batch processing, you aren’t working on live data, you are working on a snapshot, which might be fine for some companies, but for others, they need to be able to make decisions on high-velocity inbound data in near/real time.
An Overview On Wix Velo
Multi-region application architecture is a key way to solve a lot of potential headaches with globally distributed apps. It makes applications more resilien
Let’s start with the idea of a database, and a basic query. Server taps the database. Server brings back persistent state information that allows an application to update, and maybe a GUI. What’s wrong with this picture? Not much at first, of course. A GET request is the anchor of RESTful architecture, and in some ways the anchor of the web. So basic that fetch syntax defaults to that type.
A software developer scanned 2.6 million domains for exposed.env files.
MongoDB supports ACID transactions, and its cloud offering is PCI DSS compliant. It's one of the best NoSQL options for e-commerce.
Photo by Evgeni Tcherkasski on Unsplash
This is a typical exchange about encryption with someone willing to learn.
A simple tutorial on how to get started with PHP and MariaDB.
Not another Column-based Habit Tracker. Try using Relation Property to Track Habits.
In this post, I have compiled some of the most useful SQL resources that I recommend to everyone who wants to study SQL. Depending on how much time you can invest and the type of course one would prefer, I have segregated the resources into different segments. So, let’s begin:
If you have some troubles with transactions in Nest.js - this article will help you.
Technology promises change across all markets, but not all technological advances are as beneficial as they seem. In some markets, attempts at improvement end up producing inferior, complex products that cost more and don’t work as well as the product it was designed to improve.
The decision to choose a database for project is not that simple. But when it comes to choosing a database, the biggest decisions is picking a relational (SQL) or non-relational (NoSQL) data structure.
JSONB in PostgreSQL offers the best of relational and NoSQL. Here are two techniques that we use at AppLand to make JSONB data even more effective.
Object oriented database is a type of database system that deals with modeling and creation of data as objects. The main advantage of this database is the cons
Bun is a SQL-first Golang ORM for PostgreSQL, MySQL/MariaDB, MSSQL, and SQLite. It supports migrations, fixtures, and performance monitoring.
How we store and manage data has completely changed over the last decade. We moved from an ETL world to an ELT world, with companies like Fivetran pushing the trend. However, we don’t think it is going to stop there; ELT is a transition in our mind towards EL(T) (with EL decoupled from T). And to understand this, we need to discern the underlying reasons for this trend, as they might show what’s in store for the future.
If you are interested in learning SQL with MySQL database then you have come to the right place.
Golang - How to handle database migration using GORM.
Here's an example of Database testing with the MariaDB Server, one of the most popular open-source relational databases.
If you want to accelerate Trino queries with a response time of seconds to minutes, click here to learn how Trino helps engineers.
The article describes how a relational database should be designed to properly work in OLTP mode.
A trip down memory lane avid reader. Let's take a walk through the core of it all: data structures. What are they and why are they so important? A 'hello' to a reader that might have missed our talk on Memory management, where we delved into what happens to our code in variable assignment. Do take a look, even if it's a refresher you're looking for.
It's been 2 months of Firecamp's GraphQL playground launching, throughout this time I have been discussing with so many folks about their GraphQL usage and experience with Playground.
Graph databases are anticipated to surpass other types of databases, especially the still-dominant relational database.
Database migrations are driven by benefits like lower costs, better features, and the ability to scale. However, the security of data is essential.
In this article, we will be discussing the ways to optimize the database for speed in Django applications.
Another clever tool for a powerful SQL pre-processor
While the term Data is in cognizance of business of all sizes even the most layman person is aware of the buzz and fuss around Data. So from Database to Data Warehouse and now this Data Lake, we have come a long way.
In this post, I’ll explain a trick that will give your web application the ability to show friends’ posts.
I wrote a small open-source library that combines dotNetify with PostgreSQL logical replication.
In college, I discovered a new and even better way to run queries, SQLAlchemy.
ShardingSphere recreates the distributed pluggable system, connecting actual user implementation scenarios and contributing valuable solutions.
I built a Customer Support System a few days ago and started to think of a way to seed my rails database with random data for testing without having to manually type in the data. This may not be the best way to do it, but it works and allows you to test your application.
If you’re still doing manual migrations on the database you’re doing it wrong. Databases are as much part of the application as the code.
When I was recently working in one of the client project, I had to communicate with external mariadb server to store records from react/rails app, that means I would get activerecord hash from our app which I had to convert to pure sql query and send it to external server for storing.
If you have worked with sql queries previously then you must know that keys and values must be separated for insert operations likeI was recently going through my notes of SQL Antipatterns and was shocked to realize how actual this book still is.
SQL is very important and many programmers realize this, however, I have found that many more programmers just don't spend much time improving their SQL skills.
The 'varchar' and 'enum' fields are different types of data types. They are different in terms of performance, easiness to use, and support for feature changes.
A step-by-step guide 📙 on how to install, configure and run MySQL 8 server on macOS using Homebrew 🍺
MongoDB is a document-based NoSQL database, typically used for its ability to quickly scale, especially large amounts of data. MongoDB has two clear main benefits over SQL when it comes to data organization:
An exploration of AlloyDB's architecture and design, focussing on its storage engine, with descriptions of how it handles reads and writes
Learn how to set up a MariaDB database server on a Raspberry Pi 4 Model B with 8 GB of RAM that you can connect to your local network through WiFi or Ethernet
Microservices architecture patterns have highly transformed and streamlined the development process and have increased the performance of applications.
Do you know the machine learning global market is estimated to reach $30.6 billion by 2024? This marvellous growth is the outcome of Omni-presence of artificial intelligence and its trending subset; machine learning.
In this article, you'll learn how to connect to an RDS server through a bastion server in Bash using an SSH tunnel.
Imagine — You’re in a system design interview and need to pick a database to store, let’s say, order-related data in an e-commerce system. Your data is structured and needs to be consistent, but your query pattern doesn’t match with a standard relational DB’s. You need your transactions to be isolated, and atomic and all things ACID… But OMG it needs to scale infinitely like Cassandra!! So how would you decide what storage solution to choose? Well, let’s see!
We’ve just launched GraphQL Editor 3.0 and I must say it’s nothing like a year ago as we released it. When we started our vision was to provide an alternative way of designing a GraphQL schema, but what we ended up with is a lot more.
As you get into the Databases and Data Science, the first thing that you have to master is the relations between entities in your database. That is important because the data that you use has to be absolutely efficient for its further implementations.
This is going to be a very straightforward article, no jokes included.
This is a step-by-step guide for deploying a Strapi v4 project to DigitalOcean's App Platform with managed mySql Database.
GraphQL has become the most loved tool for API development in a very short span of time, and developers across the world cannot seem to get enough of it. Today I met with the founder of Windsor.io, Pranay Prakash, who has worked on the team behind GraphQL during his time at Facebook. I chatted with him to find out more about his experience working closely with Lee Byron, the creator of GraphQL, and some of GraphQL’s benefits.
Learn how time-weighted averages are calculated, why they’re so powerful for data analysis, and how to use TimescaleDB hyperfunctions to calculate them faster.
Looking for MongoDB data visualization tool? There are plenty of options but firstly its better to explore what kinds of solutions there are on the market.
This week I’m attending the 3-day Couchbase Connect event and will be reporting on some of the topics that I find most interesting.
Errors or mistakes are common in any aspects, especially in development. Using MySQL or any database can't guarantee you an error-free environment.
A valuable asset for anyone looking to break into the Data Engineering field is understanding the different types of data and the Data Pipeline.
JPA was born as the acronym for Java Persistence API.
curious about how ORM works! read this tutorial and make one from scratch using python programming language.
A lot of data over the web or internally in an enterprise is available and accessible only through Rest APIs. Rest APIs do provide a secure and authenticated mechanism to access these services, but they are not very intuitive sometimes to query upon the data which is fetched through them.
There is a lot of information about different T-SQL features. I want to tell you about equally useful, but less popular tips for working with this language!
Introducing the newest innovation from HarperDB: HarperDB Custom Functions. With the release of HarperDB 3.1 users are able to define their own API endpoints within HarperDB. What does that mean for you? HarperDB grows from a distributed database to a distributed application development platform with integrated persistence - one that can serve as a single solution for all of your backend needs. We’re collapsing the stack!
In this tutorial, you will learn how to declare and initialize Oracle PL/SQL collections (Nested Tables).
This is What I’m Telling Business People About Why Relational Databases Are So Bad
I think of web3 and decentralization on a spectrum - it’s not just one or the other, but you can take incremental steps on the path towards your end goal.
The powerful database middleware ShardingSphere core functions such as data sharding, encryption and decryption are all based on the database metadata.
Need to get up to speed on Cassandra and learn how it can benefit your software development practices? These are the essentials that cover the basics.
Introduction
Find PII data by scanning column names and/or data in your data warehouse with these two easy to create methods and a step-by-step guide on how to do it.
ACID stands for Atomicity, Consistency, Isolation and Durability and each of these have a specific purpose when it comes to databases.
Database Indexing is the most common way known and utilized by backend developers to optimize database queries.
Hello developers and enthusiasts! 😍
CockroachDB and using a popular database are two error-free options for decimal handling in Golang.
So recently I got involved with an ASP.NET project which was build over 10 years ago and over the years Developers and Change Requests came and went. And over the period the Application became quite cumbersome and quite hard to understand and manage, the Application became quite large in terms of functionality, codebase and data.
What workload analysis and running queries can teach us about the performance differences in JSON, indexing, and concurrency.
Working with model associations can be difficult.
There was an awesome debate on DBT’s Slack last week discussing mainly two things:
Announcing Appwrite 0.8 with JWT authentication, ARM support, Anonymous Login, new storage capabilities and many more new features.
Certain industries greatly benefit from high-performing, low-latency, geo-distributed technologies.
We’ve been talking about migrating workloads to the cloud for a long time, but a look at the application portfolios of many IT organizations demonstrates that there’s still a lot of work to be done. In many cases, challenges with persisting and moving data in clouds continue to be the key limiting factor slowing cloud adoption, despite the fact that databases in the cloud have been available for years.
TL;DR
This article explains what the general pros and cons of Hibernate are so that you can better determine your need to add this dependency in a new microservice.
How to make clean pagination with no additional dependencies. Basically for Node.js but can be used in any other languages and platforms.
Running systems in production involve requirements for high availability, resilience and recovery from failure. When running cloud-native applications this becomes even more critical, as the base assumption in such environments is that compute nodes will suffer outages, Kubernetes nodes will go down and microservices instances are likely to fail, yet the service is expected to remain up and running.
How to Interact with AWS (Amazon Web Services) cloud from your local machine.
Building an in-memory database in golang is a basic task that you are likely to have to do. It needs you to make a database and a table before organising.
Ok, so here is the situation: you work in a team, and each developer works with a local environment. Or you have multiple environments. Or... We don't care. The only thing is, you have to handle different versions of databases, and you don't know how to do this safely and efficiently.
That dreaded system design interview. I remember the first system design question I was asked. “Design WhatsApp”, he said. I didn’t know where to start! I was a fresher. Data structures and algorithms were the only things I knew. I am sure you can guess how that interview went. Then after enough research, I made myself a checklist of components, of sorts, to navigate me through my next system design interviews. And I sh*t you not, it works!
This article will introduce the concept of SQL recursive. Recursive CTE is a really cool. We will see that it can often simplify our code, and avoid a cascade of SQL queries!
Building Cloud-Based Data Architectures is necessary to making use of Big Data and gaining the ability to process significant amounts of data for analysis.
The purpose of this article is not to determine which database is the best but to help determine which is a fit for your specific project.
In the past few months we have been getting this question a lot:
In this article, you will learn how to set up a Heroku Postgres database with Librato for automated monitoring.
Template Queries are dynamic templates constructed with MongoDB-style operators, which allow you to customize MSON components with less code.
Both dbForge Studio and PL/SQL Developer are integrated development environments (IDEs) designed to help with Oracle database developmenttasks and PL/SQL coding productivity. However, just as any two tools,the ones we’re discussing are not exactly the same. Sometimes, it’s very important to know which solution to choose given your specific needs and preferences. In this article, we’ll look at the differences of dbForge Studio and PL/SQL Developer and analyze their strengths and weaknesses in detail.
Are you trying to decide if you should use MongoDB or DocumentDB? With the recent controversy surrounding licensing with MongoDB, it can be confusing to decide which option is right for your company or project. Amazon decided the core MongoDB code is challenging to scale while remaining highly available. Amazon wrote their implementation, which is compatible with the Apache 2.0 open source MongoDB 3.6 API. MongoDB, Inc. has also recently changed their license to make future imitations more difficult. To do this, they created an entirely new license called the Server Side Public License.
Dynamo accelerated the NoSQL revolution that’s driving the database industry.
I’ve held several roles in my career in IT, ranging from software developer to enterprise architect to developer advocate. I’ve always been fascinated by the role that data plays in our applications—putting it into databases, getting it back out quickly, making sure it remains accurate when transferred between systems. Many of the hardest problems I’ve encountered have centered around data. For example:
The InfluxBD Origin Story: How Paul Dix took a failing SaaS product and turned it into a successful commercial open source time series database company.
Introduction: The Postgres Connection Pool Problem
Graphs, and knowledge graphs, are key concepts and technologies for the 2020s. What will they look like, and what will they enable going forward?
Have you ever waited overnight for that report from yesterday’s sales? Or maybe you longed for the updated demand forecast that predicts inventory requirements from real-time point-of-sale and order management data. We are always waiting for our analytics. And worse yet, it usually takes weeks to request changes to our reports. To add insult to injury, you keep getting taxed for the increasing costs of the specialized analytics database.
An association is a connection between two Active Record models. It makes much easier to perform various operations on the records in your code. We will divide associations into four categories:
Democratizing data to enable Citizen IT provides a competitive advantage to organizations - here's why.
This article uses Exercise number 10 on the Self JOIN SQL ZOO tutorial to teach self JOINs in a beginner-friendly way.
If you’re developing web applications, you’re almost certainly going to be constantly interacting with a database. And when it comes time to select the way you’ll interact, the choices can be overwhelming.
After some time working as a data scientist in my startup, I came to a point where I needed to ask for external help with your project.
Both NoSQL databases and modern Blockchain ledgers benefit from a set of common principles. When they are both implemented for an application, a lot can be accomplished as the platforms can complement each other.
As we advance the state of machine translation, translation memory has its place in todays’ translation tech stack that benefits MT users and human translators.
The Problem with Publicly Accessible AWS Resources and how to solve it
In this article I would share a set of ORMs optimization strategies that I'm using in almost every backend service I'm working on.
The resurgence of SQL-based RDBMS
A single source of truth (SSOT) enables that synchronization. A company with SSOT relies on one and only one point of reference for the latest, aggregated info.
I’ve written about my love for Vagrant a few times already here, today I will show the steps to getting an Oracle Database with Vagrant.
AWS DynamoDB changed the database game in Serverless and continues to do so, as its design repeatedly proves its huge value.
Every business needs to collect, manage, integrate, and analyze data collected from various sources. Data integration software can help!
When you think about database providers for ASP NET Core apps, you probably think about Entity Framework Core (EF Core), which handles interacting with SQL databases. But what about the NoSQL options? A popular option for NoSQL is MongoDB. So in this article we're going to learn how to create a simple ASP NET Core CRUD API using MongoDB as the database provider.
I have been working on a GraphQL workshop, and it’s been a great learning experience. One of the trickiest things I have had to deal with in GraphQL is handling complex user permissions. It used to be a hassle until a friend pulled my attention to Hasura.
Photo by annie bolin on Unsplash
Working with a lot of data like products, orders, categories, users and payments is a very important topic when building e-commerce applications. In this post you'll learn the very basics of structuring your noSQL schema so it's fast and scalable for e-commerce scenarios.
When you enable Velo you also automatically get Wix Data, which lets you work with our built-in databases on your site. You may also want to work with data that you maintain in an external database. Velo lets you connect your site to an external database and then work with that database collection in your site just as you would with our built-in collections.
What is MongoDB?
HarperDB is more than just a database, and for certain users or projects, HarperDB is not serving as a database at all. How can this be possible?
All about MongoDB vs DynamoDB. Explore benefits, and in-depth comparison to find out the best choice for your business app.
This article will provide an overview on database architectures, including use cases and pros & cons for each of them.
For a database administrator, the common everyday practice involves running multiple operations targeted at ensuring database security and integrity. Thus, we shouldn’t overlook the importance of sensitive data stored in the database under any circumstances. In light of this, we are excited to demonstrate the new ADD SENSITIVITY CLASSIFICATION command introduced in SQL Server 2019, which allows adding the sensitivity classification metadata to database columns.
This article will be of particular interest to people who are curious about how powerful encryption can be implemented in high performance systems. It is also for people who just want to know a bit more about how strong encryption works and how all the pieces fit together.
A deep dive into the performance characteristics of RDS Proxy vs RDS
Data modeling is probably one of the most important and potentially challenging aspects of Cassandra. CQL will look familiar if you come from a relational background, but the way you use it can be very different. For our third guide, we will walk you through the process of creating a basic data model. Read part one on Cassandra essentials and part two on bootstrapping.
The founding team at HarperDB built the first and only database written in Node.js. Here's the story of this (what some called crazy) endeavor.
Latency is caused by offloading processing from an app to an external server. But what if there was a solution to the monolithic common single-cloud geography?
Learn how NoSQL differs from SQL and how it's built for high performance and availability. And no, it doesn't stand for "No SQL".
The process of checking data integrity and consistency of the database is called database testing. Ultimately, it aims to create complex queries to check database responsiveness to load/stress putting schema, tables, triggers, stored procedures, etc., under test.
Take these 10 steps to optimize your database.
Today there are hundreds of SQL and NoSQL databases. Some of them are popular, some are ignored. Some are user-friendly and well documented and some are hard to use. Some are open sourced and some are proprietary. And, perhaps, the most important - some are scalable, optimized, highly available and some are difficult to scale or maintain.
In a many to many relationship, it's just a table between the entities, but what is the purpose of this table to be between them?
The output is what measures the success of a business. Therefore, all the business houses are adopting all the inevitable methods and programs to intensify their productivity/ output by adopting new technologies and concepts. There are several technologies available and many in the developing phase, which effectively fulfil customers’ needs and generates high output.
Table of Contents
For Terrastruct, which has a little over 50K lines of code as of now, I've only written functional end-to-end tests. This blog post describes a successful setup that took some iterating to get to, and it's one I wish existed when I started with a Go API backend.
There are many ideas and considerations behind graph databases. This includes their use cases, advantages, and the trends behind this database model. There are also several real-world examples to dissect.
In this tutorial, we are going to build a web application using AdonisJS and integrate it with Materialize to create a real-time dashboard
Learn about the main differences between graph and relational databases. What kind of use-cases are best suited for each type, their strengths, and weaknesses.
Versioning database changes is as important as versioning source code. By using a database migration tool we can safely manage how the database evolves, instead of running a bunch of non versioned loose SQL files. In some frameworks like Ruby On Rails, database versioning occurs along the development. But when it comes to Java world, I don't see it happening so often.
In the post, I used a simple SQL query to explain how certain things work in SQL. I also outlined problems with the query and potential ways to improve the code
In simple terms, a database is a collection of information. It is organized to ensure easy accessibility, management, and updates.
We heard of Prometheus being an open-source solution for system monitoring, thanks to SoundCloud. But what are its other use cases? How is it being leveraged?
Get a primer on percentile approximations and why they're useful for time-series data analysis.
Integrate your external databases into your Wix sites. Learn more.
Most new developers wonder which database is suitable for their project since they don't want to get off on the wrong foot. So, first and foremost, let's define a database. A database is a type of storage facility that electronically stores and organizes data so that it may be utilized and accessed later.
Get a primer on PostgreSQL aggregation, how PostgreSQL´s implementation inspired us as we built TimescaleDB hyperfunctions and what it means for developers.
Thinking of shifting to a new database management engine? Here's how to migrate data from SQL server to PostgreSQL.
In today's economy, more and more companies exist primarily online. While there's much discussion around the consequences of leaving behind the traditional brick and mortar business, one aspect that gets less attention is the significant change in how these companies are now managing their data. Increasingly, businesses seek to understand their customers and how best to meet their needs in a way that monthly reports and KPI charts just can’t address.
Dive into the detailed history of SQL's rise and reign over the past 50 years. They remained relevant by listening and adapting to the market.
Have you ever wonder what happens inside of an Amazon warehouse? As a software engineer, the supply chain field has plenty of technical challenges.
One of the most significant problems cryptocurrencies and their blockchains face today is scalability. Unlike today’s leading blockchain networks, such as Bitco
With Velo and Wix's custom input elements, you can make extremely sophisticated forms which can do almost anything you can imagine at a fraction of the cost.
Using the aggregation functionality of the Data API you can perform certain calculations on your collection data, as whole or on groups of items that you define, to retrieve meaningful summaries. You can also add filtering and sorting to your aggregations to retrieve exactly what you need.
In this article, you will learn what a vector search engine is and how you can use Weaviate with your own data in 5 minutes.
While database technology and other tools have come a long way, it's still tough to beat the humble spreadsheet's versatility and intuitiveness. While using them for sensitive, critical applications like storing COVID-19 patient data is ill-advised, the fact that everyone knows how to use a spreadsheet means they're great for smaller cross-functional projects where a non-developer might need to examine or edit data.
How to detect, capture, and propagate changes in source databases to target systems in a real-time, event-driven manner with Change Data Capture (CDC).
With the quality of machine learning packages being developed today, testing and creating models couldn’t be easier. Data scientists can simply import their favorite library and have immediate access to dozens of cutting edge algorithms.
Android development has many functionalities depending on the developer. There are several essential components of Android app development to be executed by an Android developer and choosing the right developer for your Android app development is as important as the process itself. Three basic components most recruiters would consider is Design, Coding, and deployment of the application.
Designing a data-oriented, user-incentive mechanism is a good path when developing the future of centralised exchanges for the cryptocurrency industry.
In this article, you can find ten actionable methods to protect your mission-critical database.
Our first article covered Cassandra essentials: the “what” and “why” about the architecture. If you’re ready to try Cassandra, the next few articles will explain the “how.” At the end of these tutorials you should have a database up and running, experienced using Cassandra Query Language (CQL), gained some practical data modeling expertise, and learned best practices for building applications with Cassandra.
TL;DR
How many times have you run into a situation where you wish you could do a SQL join without getting duplicate rows back? What if we could get a list "column" returned instead? HarperDB’s ARRAY() function enables just that. In this post we’re going to take a look at a basic example of people with addresses and phone numbers.
In this article, we’ll investigate use cases for which data engineers may need to interact with NoSQL database, as well as the pros and cons.
Technical design. Because one of the most common use cases of the new databases is storing data that is generated by high-throughput sources, it is important that the store engine is able to handle write-intensive workloads, all while offering acceptable read performance. RocksDB implements what is known in the database literature as a log-structured merge tree aka LSM tree.
Earlier this month, we were joined by Joris Sijs, Lead Scientist at TNO on the second series of Grakn Orbit.
A Clinical Decision Support System (CDSS) provides the doctor with a tool that eases their work, and increases the value of the time spent with the patient.
Apache Kafka is a distributed event streaming platform built over strong concepts. Let's dive into the possibilities it offers.
Many of today’s web applications use node.js in conjunction with MongoDB to store and access data on the backend server. Not only does node.js excel in ease of access to non-relational databases like MongoDB, but brings scalability and speed to the server side of your app.
While working on a Rails application, we all have had to change the database column in some way. You can change the column name and the column type, as well as changing the column with the type conversion.
Aggregating into data lakes is the solution of today — but are Federated Sources the solution of tomorrow?
Amy Tom talks to Anil Kumar, the Product Manager at Couchbase, about becoming a Product Manager, writing his book, and creating a multi-cloud strategy.
Is database migration as scary as it sounds? Amy Tom talks to Matt Groves, Senior PM Manager at Couchbase, and Curt Gratz, Co-Owner of CKH Consulting.
Hello folks! In this article, we are going to unravel the mystery behind the Rails Active Record class. To be honest, I struggled a lot with Rails models as a beginner. I spent a lot of time reading the docs, read a couple of medium articles, watched some youtube videos but all in vain. I have chosen to draft a nice article that constitutes of baby steps that is suitable for aspiring Rails Engineers.
Redis, short for Remote Dictionary Server, is a BSD-licensed, open-source in-memory key-value data structure store written in C language by Salvatore Sanfillipo and was first released on May 10, 2009. Depending on how it is configured, Redis can act like a database, a cache or a message broker. It’s important to note that Redis is a NoSQL database system. This implies that unlike SQL (Structured Query Language) driven database systems like MySQL, PostgreSQL, and Oracle, Redis does not store data in well-defined database schemas which constitute tables, rows, and columns. Instead, Redis stores data in data structures which makes it very flexible to use. In this blog, we outline the top Redis use cases by the different core data structure types.
We’re all using resources on a daily basis. You turn on the water tap, wash your hands for at least 20–30 seconds, and turn it off. You switch the light on if it’s dark, and off when you no longer need it.
In May 2017 we made a fork of Sphinxsearch 2.3.2, which we called Manticore Search. Below you will find a brief report on Manticore Search as a fork of Sphinx and our achievements since then.
Jinq is a library that provides a convenient and natural way to build typesafe database queries in Java. It is easy to use and does not require code generation
What is SQL? Databases use it, query APIs use it, even energetic big data dashboards use it! Where did it come from and how are developers, DBAs, and app using.
It is one of the dogmas that upgrading freeware libraries is a drag - but not anymore especially for MySQL. Check out this article for more information.
Amy Tom chats to Jens Alfke about embedded databases in mobile applications and how they live within your application and sync to your server.
A story about how we implemented support for distributed transactions with no central coordinator and no single point of failure.
Jason Repp is the SVP of HarperDB, a world-leading database and development platform that is leading the charge in terms of performance, flexibility, and ease.
Check the list of the best headless cms on the market and learn more about how Headless CMS Can Help You Build Fast
bad indexing is the main source of bad performance of your database
A tutorial showing the use of SirixDB and pysirix in perserving and querying old data with ease.
Learn how to build a course management REST API with Node.js, HarperDB and FastifyJS.
Ottoman is an Object Data Modeler (ODM) for Couchbase's Node.js SDK providing JSON schema and validation for a NoSQL database.
If you are a back-end developer, you are often faced with having to migrate your database schema with each new release.
MySQL is an open-source relational database management system. As the name suggests, it is used to create, update, and store databases. It is based on SQL (Structured Query Language). So, today we will be checking out the 13 most asked questions on MySQL.
Introduction
Welcome back! In the first part of this series, we looked at a very "low-level" way to interact with a relational database by sending it raw SQL strings and retrieving the results. We created a very simple Express application that we can use as an example and deployed it on Heroku with a Postgres database.
Using data APIs and advanced data modeling will make it far easier for JSON-oriented developers to connect to Apache Cassandra.
Backing up SQL and databases using the manual restore method, the full database backup, the incremental restore, or the manageauditing command.
In this tutorial, I will show you how Materialize works by using it to run SQL queries on continuously produced nginx logs. By the end of the tutorial, you will
This story is about pain, agony, and denial of ready-made solutions. It is also about changes that improve the code’s readability and help the development team stay happy. The object of this post is an interface that helps a program communicate with a database.
With great power comes great responsibility and the bigger the project gets the easier it is to break something. Therefore you should test Postgres logic!
Recently I had to write a script, which should’ve changed some JSON data structure in a PSQL database. Here are some tricks I learned along the way.
This article looks at data uniqueness and discusses where it should be enforced. At application level or database level?
Understanding MySQL explains query output is essential to optimize the query. EXPLAIN is good tool to analyze your query.
Explore database normalization in an easy-to-understand way using pet store examples. Discover the different levels from 1NF to 3NF and the importance of data
Fauna is a distributed document-relational database delivered as a cloud API.
Dapper is a powerful and lightweight Object-Relational Mapping (ORM) tool for C#. It is designed to be simple and fast, allowing developers to execute SQL query
Build flexible GraphQL APIs by treating the Schema like a Database
What’s more frightening than Halloween? Data migration.
Harper DB is a startups of the year nominee from Denver, Colorado. Stephen Goldberg, CEO, shares the origin story and what makes their company culture relevant.
The terminal comands below will allow you to delete and re-create your Active Record database.
The DB3 Network is a start-up project to build a decentralized, permissionless platform for programmable data processing.
In this post, we are reviewing the best databases for React Native to help you create your new app.
ShardingSphere partners with Alibaba Cloud's OpenSergo to combine Database Plus & Database Mesh and release a database governance standard for microservices.
Did you ever realize that technology is changing our life gingerly? In the last few years, the world has experienced some surprising inventions and technologies. All of these have made our life faster and easier. We have jotted down some major fields where technologies bring on dramatic changes. Let’s explore how it has changed our lives.
Have you ever experienced an itch you just can’t scratch? If yes, then you will feel my pain. A few days back, everything was fine, I was happily writing code(!) and doing standup meetings regularly. Just before the weekend, my boss called me and shared this problem with me.
PostgreSQL development has built-in support for the hash table and regular B-tree indexes. The best database management system has four index access methods.
Some time ago, a friend told me that she was having trouble testing a certain application. All test solutions stressed the platform correctly, generated relevant scalability results, but at the same time they looked very artificial, she said. Well, as a self-proclaimed information security student, also known as the weird-looking-guy-guy, I could adapt some of my studies to the case.
Case study: introducing the tools used in database protocol development with a troubleshooting guide for ShardingSphere-Proxy MySQL protocol issues.
This small golang mistake could cost your business a million-dollar problem. You could easily avoid the mistake read the blog
What is YABBOS?
Taking data sharding as an example, we illustrate DistSQL's application scenarios to create a distributed database.
Rails 6 added an option if_not_exists to the create_table method which will not create the table if it already exists. This option is handy when we have added a table directly in production but now want to add it through a migration so that it is available in other environments as well.
Redis is a type of database that can be used to significantly improve your website's loading speed thanks to its design and its versatile selection of modules.
TypeORM is an ORM tool for TypeScript, supports popular databases and make it easy to interact with them. Upgrade to v0.3.11 for new features and improvements.
GraphQL Editor new release
In a world where most of the apps that we use on the internet are collaborative in nature, conflicts in data are common. Is there a way to avoid it?
Amazon RDS is the SQL Database service provided by AWS. In this blog post, learn about Amazon RDS and its feature through a comic explanation
The Firebase Realtime Database is a NoSQL database in the cloud. It doesn't have a GraphQL API, but it has a REST API that you can convert with StepZen.
Technology has taken over the world, now is the time for small businesses to realize that what they need is tech. Smart analytics makes everything easier.
Having high-quality data can make or break your projects in machine learning or business management. These 7 data quality metrics have the largest impact.
Data-driven operations are the newest frontier of the Modern Data Stack with tools.
As opposed to proprietary solutions (like Oracle), you won’t get locked in with ever-escalating license fees. Here are 6 reasons you should move to PostgreSQL.
As computing hardware became more powerful, so datasets grew. SAS in turn evolved to harness greater computational power and meet the data challenge.
I use a local password manager on my computer: I just stick with a password, which is the only password I memorize.
I had posted earlier about an open source temporal graph database that I have built, named CivicGraph.
SubQuery is a blockchain developer toolkit that makes it easier to build upcoming Web3 apps.
Is database migration as scary as it sounds? Amy Tom talks to Matt Groves, Senior PM Manager at Couchbase, and Curt Gratz, Co-Owner of CKH Consulting.
Reflections on two headless CMS implementation projects.
In hindsight, it seems like mongoose was doing the sensible thing by simply ensuring that the document existed in the database.
From simplifying data collection to enabling data-driven feature development, Customer Data Platforms (CDPs) have far-reaching value for engineers.
Poor quality data could bring everything you built down. Ensuring data quality is a challenging but necessary task. 100% may be too ambitious, but here's what y
How can we access this data correctly? Without simplifying it or opening this door, information is just bits.
Blockchains and DLTs are not the same thing. Here's why.
An extension for the Criteria API using Builder pattern and JPA Static Metamodel Generator to reduce complexity with explicitness and to increase readability
With more companies collecting customer data than ever, database backups are key.
Understanding MySQL explains query output is essential to optimize the query. EXPLAIN is good tool to analyze your query.
Evolution of our data processing architecture towards better performance and simpler maintenance at Tencent Music.
This article highlights synergy between the two widely adopted open-source projects, Alluxio and Presto.
In a world of technological abundance, it can be quite a challenge to pick the right database.
A queue using an SQL-database? well, you need to know pros and cons, and a typical implementation.
Are you using SQL engines, such as Presto, to query existing Hive data warehouse and experiencing challenges?
In this article, I will talk about how I improved overall data processing efficiency by optimizing the choice and usage of data warehouses.
I will show you how to create a database in the cloud, connect it to MS SQL, deploy from a database backup.
Refi App - A tool to make developer less painful when interacting with Firestore DB
Apache ShardingSphere & openGauss: Breaking the Distributed Database Performance Record with 10 Million tpmC.
Code or Write a REST API with GO internal libraries and MongoDB as database
A look at two databases that have made claims to the Kubernetes native label: TiDB and DataStax Astra DB.
PostgreSQL and MySQL are both reliable, secure and scalable databases that have been around for years.
Redis hashes are a type of record stored in a Redis database. They are little like JSON objects and store data as key-value pairs.
I am happy to give comprehensive tuition on building and deploying an open-source grade REST API using the Django REST framework.
Gigasheet combines the ease of a spreadsheet, the power of a database, and the scale of the cloud.
In this article, learn all about how to benchmark your DB tests.
If you are using Active Job for managing background jobs in Rails, you might have noticed that it logs the job arguments when the job is enqueued or run. Let's see an example of this.
The OLAP experience of an automobile manufacturer.
ver open up a Rails console to debug a problem and come away wondering how the data got so funky? Despite our best efforts the database will accept plenty of garbage data if you let it. There are tons of methods to bypass the Rails callbacks and validations while still updating your database. If you are like me you have probably used these methods in the Rails console to fix some of that funky data you found after some other code created it in the first place.
YikYak was an anonymous social network that used your location to show you posts 5km around you. Users of the app could create new posts and the people around them could view the posts and vote up or down.
Introduction
Compared to Dubbo, the ShardingSphere SPI is more streamlined, powerful, and easier to use. This post reviews Apache ShardingSphere’s SPI, and why it’s simple.
Benchmark a Hybrid Transactional and Analytical RDBMS (Photo: Sawitre)
Learning about best data visualisation tools may be the first step in utilising data analytics to your advantage and the benefit of your company
No need to be an expert in thousands of combinations of SQL, data types, and databases to master SQL queries. A good SQL agnostic parser will take care of all.
With cutting-edge technology emerging every year the future of database and cloud technologies look secure. Every organisation whether it big or small need database to store all the relatable data because data is more valuable than money itself.
In this article you'll learn more about the NoSQL Graph database - NEO4J.
Merging data entries from your company’s database, especially when it is coming in from multiple sour...
ShardingSphere 5.2.0 is released bringing new cloud-native possibilities, elastic migration from Oracle, MySQL, and PostgreSQL, and more features & enhancements
Tired of debugging your application to find out its database dependencies? There is a smarter way to track them with OpenTelemetry.
In this article, I covered the use of a freeware application called FDR which can help you recover corrupt or damaged MS SQL databases.
Learn about space-time indexing at scale using quadkeys and clickhouse.
Learn to use OpenTelemetry to monitor and identify the database issues in your application and remediate them to optimise your database operations quickly.
This post describes how to use ShardingSphere-Proxy and what's its differences with ShardingSphere-JDBC.
Many products solve for global issues and load balancing but unless a platform is built from the ground up with the necessary backbones, it becomes a nightmare to manage.
You must be getting unknown spam calls every single week. We often see how people get into trouble by responding to these spam calls. Hence, it is important for you to have a clear understanding about what these spam calls are and how to ensure your protection against them. Then you will be able to keep the peace of mind and ensure your own privacy.
In this article, we’ll observe another test with1.1M Hacker News curated comments with numeric fields
Learn how to integrate FlywayDB to Spring boot application to help with adding database migration functionality to your Spring Boot project using Flyway.
What are you actually missing out on in MySQL replication? It appears easy, but to debug the problem caused by it takes a lot of time. So, here's your answer.
Minimizing downtime during any deployment is a key part of any successful deployment strategy. There are many ways to achieve this.
This blog will highlight how users can define pipelines to migrate the unstructured data from different data stores to structured data via Azure Data Factory
How to connect various data sources easily and ensure high query performance.
In the past few months we have been getting this question a lot:
Wouldn’t it be great to bring the time needed to build a new data integration connector down to 10 minutes? This would definitely help address the long tail of
Explaining the Database Plus concept, its impact on system architecture design, and its innovations from Proxyless Service Mesh to microservice backend support
The 17.5 version of SQL Server Management Studio (SSMS) brought with it a new built-in security tool. Since then, the Data Discovery and Classification feature has become a difference-maker in the protection of sensitive information.
Redis is a type of database and it can be added to your production level application to make it more performant. I will cover the basics of Redis and show a real world example of Redis.
In this test we use the data collection of 1.1M Hacker News curated comments with numeric fields from //zenodo.org/record/45901.
In the past few months, I have been playing around with different kinds of IoT devices and sensors. I quite enjoy how these can be used to monitor different things like humidity, temperature, pressure among other things in the house. In this tutorial, I want to show you how you can monitor sensor readings in a database and send alerts when it crosses a threshold value using n8n workflows.
A walkthrough the process of creating a Slash GraphQL backend service to meet the needs of a fictional client, called InstaMeme.
Manage data using the HarperDB database. Access your data from HarperDB using Custom Function. Automate EDA with data from the harperDB database using sweetviz.
How one test works to analyse millions of Nginx logs from a live website and what to learn from the analysis results while processing it in a timely way.
Dapper is an open-source ORM tool that is easy to use and lightweight, which makes it a popular choice for .NET developers. It is designed to work with various
Congratulations, you’ve successfully implemented data testing in your pipeline!
Amy, Matt, and Curt talk about migrating from a SQL database to a NoSQL database, what happens when shit hits the fan, and assessing your database usage.
Amy Tom chats with Michael Nitschinger, a Software Architect at Couchbase, and Jonathan D. Turner (AKA JT), the Co-Creator of NuShell, about Shell scripting.
Amy Tom talks to Matt Groves, Senior Product Marketing Manager at Couchbase, and Rob Hedgpeth, Developer Advocate at MariaDB about their careers.
Sending mail from a Rails application has been covered by hundreds or thousands of articles, however, there is not a ton of articles about receiving, parsing and using the new ActionMailbox.
Amy Tom talks to Eric Bishard and Arun Vijayaraghavan about the differences between a Document Database and a Relational Database.
Amy Tom chats with Mark Gamble, the Product & Solutions Marketing Director at Couchbase, about Edge Computing and tiered data systems.