Building a successful partner ecosystem Pinterest ’ s Data Engineering department works with a number of third party partners , including AWS for cloud infrastructure and Percona for MySQL support , along with a number of other companies on open source software such as Netflix , Lyft , AirBnB , AWS , Starburst ( for Presto / Trino ), StarRocks Technologies , and Preset ( for Superset ), as well as close collaborations with the open source community .
Another of Pinterest ’ s partners , PingCAP , has assisted with the deployment of its TiDB system : a distributed SQL database engine that provided users with better data consistency , reducing tail latencies by 30-90 % while reducing hardware instance costs by more than 50 %.
“ We had been using an older version of HBase for many years , which is a scalable open-source , distributed , column-oriented NoSQL database ,” Burgess explains . “ We ’ ve made many fixes to HBase over the years to make it faulttolerant at our scale on AWS , used it for different kinds of use cases , and added a lot of functionality on top .”
“ The biggest pain points with this older version of HBase were : the total cost of ownership to maintain and run this ; limited functionality , which led to lower engineering productivity and increased application complexity ; the lack of data consistency across tables , affecting our users ’ experience ; and the scalability requirements our internal users wanted to run at .”
This partnership with PingCAP to use TiDB is already reaping benefits , providing better data consistency , a lower total cost of ownership , and more powerful features than the previous solution , HBase .