New Release! Git-backed Machine Learning Model Registry for all your model management needs.
We are proud to announce our SOC 2 Type 1 compliance today. This certification signals to our customers our commitment to Security, Availability, Processing Integrity, Confidentiality, and Privacy within our organization. The process to achieve this certification was rigorous. I reveal five of our key learnings from the process.
In early 2021 we determined that commensurate with our values and the importance of building best practices within the AI space, we wanted to go through the rigorous process of becoming SOC 2 compliant to ensure customers of our trustworthiness as a service provider within the AI industry. The AICPA sets out standards and certifies reports on the audits for controls in a service organization in terms of Security, Availability, Processing Integrity, Confidentiality, and Privacy. A third-party entity audits the company’s operations, policies, and infrastructure and issues a report and the certification that the company is SOC 2 compliant. This certification signifies that our company provides a high level of security, reliability, and integrity.
There are different levels of compliance. We are proud to announce that we’ve passed SOC 2 Type 1 compliance and are in the auditing process for Type 2 compliance with Prescient Assurance.
While our SOC 2 Type 2 audit is still an ongoing process, we have already come a long way over the past few months to reach this compliance. As the senior engineer leading the technical aspects of our compliance, I’d like to share some key learnings from our journey so far.
The Scope of compliance is bigger than you might think
Having had experience with GDPR compliance in the past, I assumed that SOC 2 compliance would take a similar amount of effort. This was a vast underestimation. There is quite a lengthy set of hurdles and requirements to be certified SOC 2 compliant. From setting up security incident notification processes and response plans to formulating change management policies, and setting new hires screening processes; this process affected everyone in the company to some degree.
You will learn a lot
Preparing for SOC 2 compliance is a great opportunity to gain a different perspective and get a bird’s eye view of the organization. This perspective enabled us to determine what the significant components for building a secure SaaS product are. The process forced us to examine our current practices, establish processes and policies to meet requirements, and determine and fill gaps in our existing solutions and knowledge across the team. All around it has helped us gain a clearer vision of what we do. This was not only true for those directly responsible for the SOC 2 work, but for the entire ReciprocateX team.
I ❤️ Secureframe
The third thing I found very valuable was using Secureframe. Secureframe is a SaaS platform that helps prepare a service organization for the audit. It has a list of integrations, tests, and reports built into the platform, which provide you with enough details for the work to be done. I highly recommend using this tool if you are taking on SOC 2 compliance.
There are lots of intricacies to making infrastructure secure
During the preparation process before the audit, we spent a lot of effort tuning our infrastructure from a security perspective, to make it match according to defined policies and terms. I faced some interesting cases where enabling MFA to access parts of our infrastructure was not air-tight, and we had to resort to non-elegant workarounds.
We are using EKS in AWS cloud for our services, and based on the Access security policy, every employee should have fine-grained multifactor authorization access. This means that if you don’t provide an OTP you cannot access the Kubernetes resources via the ODIC provided by AWS.
So for implementing MFA we have an IAM policy that is attached to every user which blocks access to the resources.
statement {
sid = "DenyAllExceptListedIfNoMFA"
effect = "Deny"
not_actions = [
"iam:CreateVirtualMFADevice",
"iam:EnableMFADevice",
"iam:GetUser",
"iam:ListMFADevices",
"iam:ListVirtualMFADevices",
"iam:ResyncMFADevice",
"sts:GetSessionToken",
"iam:ChangePassword"
]
resources = [
"*"
]
condition {
test = "BoolIfExists"
variable = "aws:MultiFactorAuthPresent"
values = [
"false"
]
}
}
But for authenticating in Kubernetes there is a tool named
aws-iam-authenticator
which allows us to use IAM credentials for it. Under the hood it also uses
sts:GetSessionToken
to retrieve the needed token from AWS.
The problem we found was that AWS IAM sts:GetSessionToken
does not support
passing custom condition keys, so that meant the users were able to access
Kubernetes without MFA, thus taking us out of compliance. To solve this we
started to use an IAM role which grants necessary access. But interestingly,
the users who created the Kubernetes account still had access. To get around
this, the creator user account can be disabled or we can use disposable
accounts to create EKS clusters, both pretty ugly workarounds.
Vulnerability Scanning opened a can of worms
Regularly performing vulnerability scanning is a requirement to fulfill SOC 2 compliance. We incorporate a couple of different tools for this: AWS Inspector and Grype. Both tools surely provided these vulnerability checks, but also presented some new challenges with which we still need to deal.
First, the two tools provide disparate vulnerability reports. This introduced an extra level of comparison and a deep dive into the vulnerabilities to understand the extent of them and the disparity between the reports. Second, there are, quite literally, thousands of low-level vulnerabilities that come up in those reports, most of which we have no ability to change because they are under the control of other open-source libraries. The breadth of these vulnerabilities and our inability to change them opens up the concern that we may stop paying attention to the reports and miss a true vulnerability that we could do something about.
We are still brainstorming ideas on what to do about this issue. One idea is to include a vulnerability report with each of our releases so that we can review and fix what can be corrected and provide transparency on those that we cannot change.
The process was full of ups and downs, but in the end, the gains to our company, our team, and our customers are proving to be well worth it. Please visit our Security and Privacy page to learn more about our policies, and access the SOC 2 Type 1 report.