Your selection of data warehouses will bring in a healthy return on investment (ROI). Nevertheless, the ROI of any data warehouse will considerably rely on its features and how well it integrates with your current business workflows and processes. Therefore, concentrating on functionality will need to be prioritized compared to focusing mainly on pricing.
That being said, the cost is still a vital factor in any data warehouse purchasing decision. In this post, we’ll walk you through data warehouse costs so that you can reach a smarter decision for your business.
What can a data warehouse give you?
A data warehouse deals with storing your enterprise data, collected from a diverse range of sources, in an organized and effective manner.
This data warehouse is then combined with a business intelligence tool, allowing users to uncover trends and implement sophisticated analyses. Your data warehouse makes sure the information can simply be queried by your selection of BI solutions.
A data warehouse is different from a database. ETL tools gather information from various locations, covering databases and other sources like websites and files, and put it in a single centralized data warehouse. The ETL process (extract, transform, and load) can occur automatically or manually under scheduled pre-defined conditions. When data is in the warehouse, it can then be configured and manipulated.
Is your structured and unstructured data widely distributed throughout your business? Let’s build a data warehouse to solve the problem.
Data warehouse costs: things consider
Storage
The first thing you need to determine is whether you want to store your data warehouse in the cloud or on in-house servers. If your business already uses a scalable server solution, you can extend some of your storage to fit a data warehouse.
In-house storage
Storage fees for an in-house data warehouse can go up to $12,000 per month. In the current marketplace, each additional terabyte of data will charge you upwards of $1000 per year in storage and access fees.
Cloud storage
If your business is looking for a faster and more scalable data warehouse storage option without any up-front hardware spending, cloud-based storage is the go-to choice.
Almost all analytical storage solutions require an additional fee for accessing the data on their servers. Every query and event will cost you: BigQuery requires an additional five cents per gigabyte, while Azure requires five cents for every 10,000 rows of data.
Here are three of the most popular storage solutions available on the market:
- Amazon Redshift
Amazon’s hot storage solution charges $0.08 per gigabyte (GB). This is a flat cost, which means the prices won’t increase or decrease based on your data use. However, it’s still expected to pay more than $1000 per terabyte (TB) per year if you use Amazon.
- Google BigQuery
Google’s storage solution charges $0.02 per GB. This is a variable cost. Google also requires an extra price for accessing this data. You will need to pay an additional $0.05 for every 100GB that the service accesses for you. If your data warehouse is often accessed by your analytics team, the prices can go up. It’s expected to pay over $720 per TB per year if you go for BigQuery.
- Microsoft Azure
Azure offers pricing in blobs, meaning the prices increase when you store more data on Azure’s servers. For the first 50 TB per month, you’ll pay $0.0184 per GB, and then the prices go up. Moreover, Azure requires access prices, meaning an extra $0.05 for every 10,000 write operations. Therefore, this solution will cost more than $700 per TB per year.
| Microsoft Azure | Google BigQuery | Amazon Redshift | |
|---|---|---|---|
| Per TB per month | $18.43 | $20.48 | $81.92 |
| Per event | $0.05/10K Queries | $0.05/100GB | N/A |
Software
Most software with data warehousing functionality is called ETL software.
Although ETL can be implemented via certain open-source solutions such as Talend and Apache Hadoop, let’s consider some software options that apply ETL to integrate common database types and SaaS platforms into your data warehouse.
Data centralization
There are some options for centralizing your data on cloud storage services. Most of the existing software integrates easily into Google BigQuery or Amazon Redshift.
- FlyData
FlyData is a comprehensive solution that integrates your databases with your Amazon Redshift storage. The solution enables VPN tunneling and error solving, and different database formats like MySQL, TSV, CSV, JSON, and PostGRESQL.
FlyData does not come with any visualization solutions, so it can be used along with services like Tableau and Chartio. Its pricing varies from $200 to $2000 per month.
- RJMetrics
RJMetrics provides powerful support for SaaS problems such as Google Commerce, Shopify, and Salesforce. In addition to supporting the same databases as FlyData, this solution also supports Microsoft SQL Server, MongoDB, and Heroku. Its pricing ranges from $500 to $2000 per month.
- Fivetran
Fivetran handles all data from your databases and SaaS services into Google BigQuery and Amazon Redshift databases. However, its pricing model is unavailable online, so you need to contact Fivetran for more information.
| Fivetran | RJMetrics | FlyData | |
|---|---|---|---|
| Less than 10 people | $200-500 | $500 | $200 |
| More than 10 people | N/A | $2000 | $2000 |
Data visualization
While some business analysts only work with open-source visualization, you might want to consider some data visualization services that can handle your team’s workflow automatically.
- Chartio
This business intelligence tool creates visualizations in real-time. The service provides a drag-and-drop interface that creates SQL queries for you. Its costs depend on the size of your business, ranging between $500 and $2000 per month.
- QlikView
QlikView provides pricing per user, beginning at $25 per user per month. You’ll pay upwards of $250 per month for a 10-member team.
- Tableau
Tableau comes with a user-based pricing model, with $70 per user per month. If you have a team of 10 members, you need to pay about $700 per month.
Total software cost: With a conservative medium price for both visualization software and ETL, your projected costs will stay around $2000 per month or $24,000 per year.
Bear in mind that this is an initial estimate. When beginning to start your own in-house data warehouse budget, pay attention to the following:
Your software prices certainly increase over time. Extended storage comes with extended ETL usage and higher storage access prices, boosting the cost of your data warehouse every month.
Pricing models are unpredictable, based on the size of your storage, the amount of data you put into your warehouse and the size of your team.
Human resource
For your database support team, you will need to have an enthusiastic Systems Manager, Backend Developer, and Software Engineer. These people will ensure that your data warehouse is operating seamlessly, and that all data is manageable and safe.
For the visualization team, you need to have at least one Data Analyst.
Here are some job descriptions and conservative cost predictions connected with staffing an entirely operational data warehouse. The salary ranges were measured using the average income of every respective position on payscale.com.
- Information system manager: An IS manager needs to supervise the team implementing your warehouse and keep all the systems in check. It’s estimated to pay about $7000-10,000 per month for a qualified IS manager.
- Backend developer: Backend developers are in charge of installing and maintaining all of your ETL software and ensuring that it is working properly with your storage service. It’s expected to pay about $6000-8000 for backend developers per month.
- Data architect (DBA): A DBA will decide the structural requirements of your data warehouse and suggest the best solution for merging all of your current data sources into it. It requires around $10,000-12,000 per month for a qualified DBA.
- Data analyst: Your data analyst will need to analyze and visualize your business intelligence in a way that generates meaningful insights. A data analyst will cost you about $5000-8000 every month.
An estimated cost for a data warehouse
Having all of the necessary components to create your data warehouse, let’s see how much the total is. Suppose that you want to have a data warehouse that will require one terabyte of storage and 100,000 queries per month, your total annual cost for storage, software, and staff will stay around $468,000.
Remember that this pricing ranges depending on each business, each industry, and each period. It’s impossible to estimate the exact cost of your data warehouse.
In case you do not exactly understand what is going on and how to choose the most suitable solution for your business, we got you covered! Synodus Data Analytics Consulting Services provides you with a wide range of expertise, technologies, and consultation. Our seasoned experts could lend you a hand and check-up to determine which way is the best data warehouse solution for your business.
Closing words
The imprecise prediction of data warehouse costs can make your project end up with overspending and slipping behind schedule.
The cost components mentioned above can be your first step to prevent a negative outcome. For a more accurate estimation, you should consider many more aspects. If you’re not sure you get all the factors to predict your data warehouse project cost, Synodus’s data analysts can help you determine them and bring you an estimation. Let’s have a chat!
