This project proposes a framework for an automated method to measure online deliberation by analyzing commentary surrounding a complex social issue. By employing a combination of analytic calculations, alongside tools such as Google Perspective and LIWC, we assess the sentiment and quality of online deliberation using established measures of deliberation including rationality, interactivity, equality, civility and cognitive complexity. We test our automated approach via a case study analyzing Twitter and Reddit commentary related to convoy protests surrounding COVID-19 vaccine mandates and restrictions. These protests took place in Ottawa, Canada, during February, 2022, and served as an inspiration for similar protests internationally. In addition to testing our proposed online deliberation measurement framework, this case study provides insight into the deliberative characteristics of the Twitter and Reddit social media platforms.