My team doubled in size this year, and with that kind of growth, it’s easy for costs to get out of hand when you’re using usage-based platforms like Databricks. But despite more users and more workflows, we’ve kept our total cost of ownership low. We did this by being intentional — we’ve kept everything self-contained inside Databricks, avoiding expensive third-party ETL tools, and leaning on open-source where it makes sense. Here’s what’s worked. ...
I have been using Frigate for a couple years now as my NVR and for camera alerting. It’s been great to use but also frustrating at times to set up. It’s gotten better over the years, but it’s still not user-friendly and definitely not something you should set up for your parents or grandparents. I used to run Frigate in Unraid, but I recently dropped it in favor of TrueNAS for performance reasons. With this, an opportunity arose to migrate Frigate into my ProxMox virtual environment. The benefit of doing this would mean I can easily back up my frigate environment, I can migrate it to another machine if it fails, and I can easily test configs without worrying about losing my current config. ...
One of the most interesting features Databricks has released recently which is now in public preview is Databricks Apps, a new way to host and share interactive tools directly within the platform. I’ve had the benefit of testing Databricks Apps since last summer during its private preview. At that time, there was no UI; everything needed to be deployed manually as code. Since then, there have been many quality-of-life improvements, and I feel it’s in a good enough state now to start talking about where Databricks Apps shine, and when you might want to reconsider using them. ...
I just wrapped up CSE 6242 – Data and Visual Analytics at Georgia Tech this past semester. I finished with an A, but I’ll be honest: it was one of the more difficult courses I’ve taken in the OMSA program. Between unclear expectations, inconsistent grading, and group work that felt more like coordination overhead than learning opportunity, it took quite a bit of effort to get through. If you’re planning to take this course or currently in the thick of it, here are a few thoughts and strategies that helped me stay on track. ...
I have been running a hodgepodge homelab with a mix of enterprise hardware and consumer-grade mini PCs, and it has been challenging to monitor all of my hardware directly within PVE. I am fortunate that my wife lets me run wild in the utility room, but when the OPNsense router goes down, I end up with an upset wife and no internet — which is understandable since she works from home. ...
Having worked in data and analytics for over 10 years, and having built two different data practices from scratch, I think I can safely say I have a good sense of what makes a data practitioner successful. Earlier in my journey, many of my peers and team members came from industry, ie. people looking to shift their careers and pivot into data. However, over the past four or five years, that momentum has really picked up, and many of my hires have come directly out of college. ...