The first step to controlling costs is understanding what’s happening financially in your cloud data: who’s spending what, where, when. You might have thousands of data engineers spread across the globe, and they’re all spinning up instances—incurring costs—to the tune of 10,000s of jobs a month. Each of these carries a price tag, and each is an individual “spending decision” about how much it’s going to cost to run in the cloud.
So first you need dashboard-level insights (ideally, with drill-down capability) into who’s spending how much, how often, when, and on what.
Through the combination of tagging and data observability, a DataFinOps approach uses cost-allocation data to visualize cloud data costs from a variety of business perspectives:
- By individual user
- By individual job
- By team, department, LOB
- By project, initiative, or “data product”
- By application/pipeline
- By cluster
- By environment (Dev vs. Prod)
- By budget
Allocating costs is foundational to the DataFinOps principles of making collaborative data-driven spending decisions and holding individuals accountable for their cloud usage/costs.