Solutions Architect

Role overview

This Solutions Architect leads the design and deployment of AI-powered applications on the Azure Databricks platform. The role focuses on architecting solutions that run natively on the Databricks serverless environment, removing the need for separate hosting infrastructure while integrating tightly with core platform services for data governance, querying, authentication, and model serving. The position blends hands-on technical architecture with cross-functional leadership across data, security, and application teams within a higher education consulting context.

Responsibilities

- Lead end-to-end deployment of AI applications on the Databricks Apps platform, from architecture design through production rollout and ongoing maintenance. - Architect serverless-first solutions that inherit the platform's built-in security, compliance, and resource management instead of relying on external hosting. - Design and integrate AI agents using Retrieval-Augmented Generation patterns and Databricks Agent tooling, managing the full lifecycle from creation through deployment, testing, and monitoring. - Establish data governance through Unity Catalog for centralized access control, lineage, and security across application data assets. - Enable efficient data access by integrating applications with Databricks SQL for large-scale dataset querying. - Implement secure authentication using OAuth and platform-native identity and authorization patterns. - Configure and optimize Model Serving and Serving Endpoints for deploying and scaling ML models and LLM agents. - Collaborate with data engineering, security, and business stakeholders to translate requirements into scalable, production-ready solutions. - Define architecture standards, reference patterns, and best practices for AI application development on Databricks.

Requirements

- Proficiency with Databricks SQL for designing and optimizing queries against large datasets. - Experience implementing secure authentication and authorization flows using OAuth and IAM. - Hands-on experience deploying and scaling ML models and LLM agents through Model Serving and Endpoints. - Familiarity with RAG architectures and AI agent frameworks such as LangChain and/or LlamaIndex. - Experience using MLflow for logging, testing, debugging, and monitoring agents and models. - Proficiency with application frameworks such as FastAPI, Flask, Streamlit, Dash, or Gradio, plus front-end experience with React or a comparable modern JavaScript framework. - Strong Python skills for application, agent, and pipeline development. - Experience with Databricks Jobs and workflow orchestration for data pipelines and ETL. - Working knowledge of Microsoft Azure networking, security, and compliance fundamentals. - Strong architectural judgment balancing scalability, security, governance, and cost, with the ability to lead technical delivery and mentor engineering teams.

Nice to have

- Prior experience delivering enterprise chatbot or conversational AI solutions. - Familiarity with token streaming, request logging, and review-app patterns for production AI agents. - Industry-specific solution experience in operations management or analytics-heavy domains. - Relevant certifications such as Databricks Certified Data Engineer/ML or Azure Solutions Architect.

Benefits and work setup

This is a 1099 project-based contract consultant role with remote flexibility. Consultants partner with higher education clients, collaborating with functional and technical teams to evaluate current processes, design future-state optimal processes, and architect, develop, test, train, and roll out solutions. The arrangement offers flexible assignments suited to independent professionals, exposure to varied projects across diverse client needs, and opportunities to grow a portfolio while making an impact.