I then asked if it was fine or if I was going to find corruption when I tried to read it, and they suggested I do db.repairDatabase() to rebuild everything. No sooner said than done, I ran the command and left it for a few hours to run. At some point I lost my SSH connection to the server but thought that it would probably keep running or, at least, leave the database in some half-consistent state so I could rerun the query. That’s exactly what it did, but it also deleted 90% of my data, which I couldn’t recover.

In summary, I would not let MongoDB near my children, let alone run it on a production environment (or even a testing one). It’s a great piece of software when it works, but that’s not very often. A database that doesn’t really store data is very, very dangerous.

The datastore is often the most important part of an application. Code can be changed easily, and new code can be deployed without much fuss if you discover that some of your original choices were wrong, but the data model and the way it is handled is much harder to change. This means that you need to give the data model as much thought as you can when starting out, and the choice of datastore greatly influences that decision.

This post is meant to guide you through some common pitfalls, and hopefully explain why a relational database is a much saner default than the schemaless databases I see most people instinctively reach for nowadays. //

Schemas are awesome. More importantly, they’re inevitable. There’s no application that doesn’t use a schema, as there’s no application where you only write data without the reader needing to know what kind of data it’s going to read. The schema is just implicit, in the application code, instead of explicit, in the datastore, where it should be. This means that schemaless databases aren’t really schemaless, they just kick the can down the road and let you get away with not defining a schema early on, which invariably comes back to bite you in the ass later.

When you use a schemaless database, you’re essentially saying “I don’t want to deal with the schema now, just let me write the data, my future self can deal with it when reading”. However, when would you rather get an error? When you write the data in the database and can retry, or a year later, when you read it and it turns out that that one entry had escaped notice and is still using the old format?

In almost all cases, the choice is clear, you want your application to produce an error while writing, when you can still do something about it, and ensure that your datastore will always contain “clean” data. That’s impossible to do without a datastore that enforces a schema.

Within modern software development practices, you are expected to build software from a two- or three-sentence business feature. The name of the game is to not go too deep and to constantly iterate. This is far different from the traditional waterfall method, where you would spend six months figuring out the requirements analysis before you wrote a single line of code. With a waterfall approach, this is fine because you will know the end state in order to create your database objects. However, you simply can’t do this if you’re following agile methods because there is no way to build a data model from a three-sentence business requirement, and you’re constantly having to rework your database. //

RDBMS Growth at Travelers
YEAR Tables in production
2013 40
2015 70
2017 100
//

We taught every team how to data model in JSON, as opposed to the rigid tables and rows of a relational database. This was an eye-opening experience for many people, who now understood how this affected the speed at which our teams could deliver software into production. //

At the end of the day, if you’re doing it right in relational, you have a lot of tables, and if you’re modeling your data properly, you will have a lot of objects even for the simplest use cases. Once we determined that our database was slowing us down, we knew it was time for a change.

DB-Engines is an initiative to collect and present information on database management systems (DBMS). In addition to established relational DBMS, systems and concepts of the growing NoSQL area are emphasized.

The DB-Engines Ranking is a list of DBMS ranked by their current popularity. The list is updated monthly.

The most important properties of numerous systems are shown in the overview of database management systems. You can examine the properties for each system, and you can compare them side by side.

In the database encyclopedia terms and concepts on this topic are explained.

The Database Built for Speed and Scale
SingleStore is a distributed, highly-scalable SQL database that can run anywhere. We deliver maximum performance for transactional and analytical workloads with familiar relational models.
The Free Version Includes:

Full developer and administrative access
Up to 4 nodes
128 GB memory capacity
Ability to run on-premises or any cloud
Guided tutorials, self-paced exercises & more

SingleStore is The Database of NowTM for Modern Applications. Run both transactional and analytical workloads at scale with an integrated, familiar, durable SQL database

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

You can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing an element to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set.

In order to achieve its outstanding performance, Redis works with an in-memory dataset. Depending on your use case, you can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log. Persistence can be optionally disabled, if you just need a feature-rich, networked, in-memory cache.

Redis also supports trivial-to-setup master-slave asynchronous replication, with very fast non-blocking first synchronization, auto-reconnection with partial resynchronization on net split.
Redis is written in ANSI C and works in most POSIX systems like Linux, *BSD, OS X without external dependencies. Linux and OS X are the two operating systems where Redis is developed and more tested, and we recommend using Linux for deploying. Redis may work in Solaris-derived systems like SmartOS, but the support is best effort. There is no official support for Windows builds, but Microsoft develops and maintains a Win-64 port of Redis.

the MySQL developers actually messed things up with UTF8 and had to add 4-byte support afterwards(!) and even according to the MySQL documentation they describe possible issues when converting (UTF8 should just be UTF8. Convert UTF8 to UTF8? Come on!) from UTF8 to UTF8mb4. It sounded bad, and almost not true. So we decided to read about it ourselves, and as it turns out the IRC guys were right. PGSQL is a more advanced database in general and handles stuff more efficiently than MySQL/MariaDB, which also is proven in different benchmark testsSo we decided to give it a try, made a new branch, and started developing a PGSQL VM. We were kind of amazed since the end result was removing a lot if MariaDB lines and replace them with just a few PGSQL lines,Not only that, PGSQL is faster, it actually flies! Folders with many files loads faster, a gallery with many images loads faster,it also gives a feeling of being more secure. No need to save the root password in a separate file like we did with MariaDB, with PGSQL you authenticate with the UNIX user. Yeah you can do that with MariaDB as well, but again, it’s a feature, PGSQL does that by default. If you are convinced to change then great! The process is very easy and Nextcloud even have built in support for converting SQLite / MySQL / MariaDB to PGSQL (or the other way around) with the occ command. To make the leap smaller and easier for you we made a script that we used ourselves to make the change. Remember to test it on a clone of your server before you do it in production, you never know if and when stuff go wrong, and it’s better to be safe than sorry.