As the service owner, I am super excited to share that Azure Defender for Key Vault is now generally available! It is really One Microsoft experience to work closely with Azure Security Center and Azure Key Vault team to launch this service. Also personally, I grew up a lot after going through the Machine Learning algorithm improvement, infrastructure refactoring, BCDR and privacy policy compliance, cost reduce, monthly business review(MBR), customer feedback investigation. It is indeed a challenging and inspiring work to wake me up every day.
Azure Certifications and Exams
Congratulations to myself! I just hit my July goal and am now a certified Azure Data Engineer! Do you also want to snag some cloud certifications? Excellent! You've come to the right place!
Spark stateful streaming processing is stuck in StateStoreSave stage!
A stateful structured stream processing job is suddenly stuck at the 1st micro-batch job. Here are the notes about that issue, how to debug Spark stateful streaming job, and also how I fix it.
Databricks Migration Guide
When you need to migrate an old Databricks to a new Databricks, all of the files, jobs, clusters, configurations and dependencies are supposed to move. It is time consuming and also easy to omit some parts. I document the detailed migration steps, and also write several scripts to automatically migrate folders, clusters and jobs. In this chapter, I will show you how to migrate Databricks.
Always flush Application Insights
Databricks docs says, the SDK sends out data at fixed intervals (typically 30 secs) or whenever the buffer is full (typically 500 items). However, from my personal experience, it won't send the data if you don't flush.
Java serialization is a bitch!
There is no reason to use Java Serialization anymore.
Deploy Spark .NET app on Databricks
I struggled to deploy a Spark .NET app on Databricks scheduled by Azure Data Factory pipeline. Here are the notes on the solutions how I finally figured out. From this chapter, you can step-by-step create a Spark .NET app and deploy it either on Databricks directly or scheduled by an Azure Data Factory pipeline.
搭建Hexo + Github Pages + Travis CI个人站点的详细教程
本文详细介绍了如何快速的搭建Hexo + Github Pages + Travis CI个人站点。以Next主题为例，介绍了在项目中添加了作为git submodule的主题后，如何正确的部署站点和发表文章。以及如何使用Travis CI将Hexo项目自动部署到Github Pages。