Process Quality Reviews (PQRs) are security assessments that scrutinize major key elements of deployed DeFi products to produce an overarching safety rating that indicates the quality of development processes and accompanying documentation.
The premise behind these reviews is that developers following best practice in software development should have secure code and maintain security that users can trust, as potential blind spots have been accounted for.
DeFiSafety’s PQRs are conducted according to a clear, standardized and replicable process. This process is well-documented and publicly available, so that readers can dig into the details of the review to see how the score was generated.
Written in clear and accessible languages, PQRs serve to build confidence in DeFi, raising standards of security and transparency, and providing value to all members of the DeFi community.
Clear, concise and detailed ratings give users a more comprehensive understanding of the protocols they’re investing in. This allows users to make informed decisions regardless of their technical expertise, and helps build public confidence in DeFi as a whole.
Clear guidelines on improving PQR ratings gives developers achievable targets to improve code quality, transparency and documentation, and raises the bar across DeFi.
PQRs incentivize the use of more comprehensive and high-quality audits to achieve a better rating, and disincentivize the use of fake or trivial audits. Additionally, encouraging clear and accessible documentation facilitates auditor mandates.
The PQR process scrutinizes five key elements of deployed smart contracts and rates them according to how closely they follow best practice.
We review the development history of the code in the application and top-level transparency indicators; whether developers are published or anonymous, whether code is public or private, and whether smart contract addresses you interact with are publicized and easily located.
While transparent systems are not guaranteed to be free from exploits, they allow anyone to review them in real time and search for potential problems. This increases the capacity for community monitoring, and is an indicator of how serious a team is about their design process.
High scores are given to projects with clear guidance to smart contracts, public software repositories, actively used code, clear development history and teams with public members.
We verify the quality of system and software documentation, as well as accompanying commentary.
Good documentation indicates time spent on developing protocols to interact with the DeFi ecosystem, allowing other developers to easily integrate their own protocols and effectively interact with the smart contracts themselves. It also proves teams are able to onboard new developers effectively, and demonstrates a commitment to developing effective protocol foundations.
High scores are given to projects with a white paper describing the premise and function of the application, comprehensive software documentation and commentary, and traceability between software documentation and implementation.
We verify the test suite used to validate the application before deployment to the mainnet.
Thorough testing implies that the team has taken the necessary time to develop their product, and demonstrates that code is battle-tested. It proves that code works the way it's supposed to, and ensures against dead code and vulnerabilities. It demonstrates adaptability, and the ability to quickly and effectively implement and validate changes when necessary.
High scores are given to projects with a high test-to-code ratio, with testing covering all the deployed code, instructions for 3rd parties to run tests, reported test results, formal verification and proof of stress testing.
We verify the presence and quality of third-party software audits and bug bounties.
Good security means that protocols are more difficult to hack. Code with a high security score has been audited multiple times, and offers a financial incentive to would-be hackers to submit bug bounties instead.
High scores are given to projects with one or more publicly reported, comprehensive and high-quality audits, as well as a high value bug bounty program.
We verify documentation and disclosure of special admin access controls to the protocol.
Good access controls mean that developers are transparent with the control that they have over the protocol, and demonstrates the upgrade power of a protocol. It also demonstrates that the protocol is able to pause their contracts if problems occur with the contract (hack, bank run, etc).
High scores are given to projects with clear and complete information regarding access controls, and immutable (unchangeable) contracts.
This section goes over the documentation that a protocol may or may not supply about their Oracle usage. Oracles are a fundamental part of DeFi as they are responsible for relaying tons of price data information to thousands of protocols using blockchain technology. Not only are they important for price feeds, but they are also an essential component of transaction verification and security.
Since Oracles play such an important role in the space, it is vital that protocols disclose the complete extent of their Oracle usage. Such information includes the provider, the refresh rate of price feeds, which smart contracts are using Oracles, the Oracle’s key software functions, and more. The availability of this documentation is not only important for developers, but also for users who want to get a good grasp on the robustness of the Oracle used by their favourite protocols. In an era where Oracle manipulations are rampant, it becomes increasingly crucial to prioritize them for the sake of best practices and transparency.
The author of this process has a long history in the avionics industry. Aerospace software development has always maintained that a rigorous and frequently audited software development process leads to safe and secure software that can be supported for many decades. The avionics process is DO-178C. It is significantly more rigorous than the expectations of this review process, but nevertheless informs the overarching philosophy that guides the specification of this review process.
What we do is quite different from smart contract auditors in a number of ways. Smart contract auditors are paid by the developers to check their code. The report is written for the developers (who paid for it) and generally written in technical language. The developers set the scope of the audit both in time and subject. The developers decide if they want to publish the report. Effectively, all the power remains in the hands of the developers.
Our target audience is the users of an application. While developers gain value from what we do, we do not ask their permission before doing a review.
PQRs are a way of quantifying the degree to which a development team adheres to best practice in the development of their protocol.
High scores indicate that a protocol follows best practice in terms of development, documentation, testing, security and transparency, and can be considered trustworthy and safe because the potential risk vectors are being accounted for and guarded against. It is exceedingly difficult to scam users when your code and access controls are completely public, transparent, tested, and subject to comprehensive audits by reputable firms.
High scores do not indicate that a protocol represents a good financial investment. Instead, they indicate that users of that protocol are much less likely to be scammed or exploited due to oversight or malicious intent.
In contrast, low PQR scores indicate that users of a protocol may be leaving themselves open to exploitation because some aspects of the protocol deviate from best practice, which may translate into unrecognized vulnerabilities.
Low scores do not indicate a malicious or untrustworthy development team. Instead, they indicate that a protocol has preventable blind spots through which a hack or scam might occur. In theory, a highly trustworthy and exceedingly competent development team could create a protocol with bulletproof code and still receive a low score due to inaccessible documentation and insufficient auditing.
Some examples of what a PQR score can be used for: