However, Software-as-a-Service (SaaS) utilisation was found to be on the lower-end of adoption, with just 18% of respondents stating the use of such software, which Benchling suggested indicated a high level of legacy systems still being in use across the industry.

Pharma dips toe into tech adoption, but is it deep enough?

Trending: Kevin O'Leary-Backed Startup Lets You Become A Venture Capitalist With $100 Benchling is at the forefront of harnessing the transformative power of biotechnology, which is reshaping ...

These Biotech Companies Are Poised For Big Things In 2023 And Beyond

Biotech companies continue to live up to their hype, with many of them reaching new heights in 2023. But that doesn't mean they've peaked. There's plenty more room for growth throughout the rest of the year and beyond.

During this session, Adrian will provide a concise scientific overview of the formulation process before delving into the ways his team leverages Benchling for effective experimental data management. Gain insights into common pain points and problem ...

Unlocking Data Insights on Benchling for Novel RNA Therapeutic Delivery Methods

Life sciences-focused cloud startup Benchling has seen its business soar over the last few years, amid a renewed interest in biotech among investors. Benchling CEO Sajith Wickramasekara joins Caroline Hyde and Ed Ludlow to discuss the state of the global ...

Biopharma's Tech Adoption Challenges

Benchling has launched its inaugural 2023 State of Tech in Biopharma report, which has shed light on the obstacles that biopharma encounter when striving to fully implement and embrace these technologies. The report surveyed 300 R&D and IT experts from ...

2023 State of Tech in Biopharma report reveals tech strategies in era of data and AI

Report indicates a complex tech environment for R&D and IT, key barriers to address, and investment priorities for next 12 months  LONDON, September 21, 2023 — Today, Benchling launches its inaugural 2023 State of Tech in Biopharma report.

2023 State of Tech in Biopharma Report Reveals Tech Strategies in Era of Data + AI

Native instrument connectivity, open industry data standards, and built-in scientific metadata are key to unlocking instrument data at scale in Benchling Connect Biotech has long been in gridlock ...

Introducing Benchling Connect: Tackling Lab Instrument Connectivity and Data Management with New Open Standards and a Flexible Platform

Benchling Bioprocess delivers new design, planning, and execution capabilities on a unified platform, to address the evolving requirements of modern process development Process development teams ...

Announcing Benchling Bioprocess, the End-to-End Solution for Modern, High-throughput Process Development

Benchling

benchling

Hey Hackers, has been nominated in HackerNoon's annual Startup of the Year awards in San Francisco, USA. NExTNet Inc. Please vote for us . here Read more about us below to understand why we deserve your vote. Are you also participating in SOTY 2023? If so, click here to fill out this interview. Meet NExTNet Inc. is a Venture-backed Silicon Valley-based Enterprise software startup developing Generative AI technologies for mission-critical applications in biotech and pharma. Our mission is to organize and integrate the world's biomedical knowledge and make it accessible. Imagine NExTNet = Oracle x Palantir. Meet our team . NExTNet here My Role In late 2017, I founded a fast-growing biotechnology platform company developing ground-breaking silicon chip technologies to accelerate the development of personalized medicine. Imagine Nvidia for biotech. I founded in late 2020 frustrated by the extremely high technical barrier to accessing the scattered knowledge buried within mountains of multi-modal and disparate biomedical data and information. Our goal is to democratize access to the world's biomedical knowledge to accelerate drug discovery and development. There are more than 23,500 diseases known to mankind, of which ~3% have some form of cure or treatment. Mekonos, NExTNet How We're Disrupting the ‘ Industry Intersection of AI and Biomedical Roughly 95% of the world’s data have been generated in the last 5-10 years. The emergence of high-resolution, multi-modal biomedical data (10s of millions of scientific publications, patents, grants, sequencing data, gene and protein expression, drug compounds, biochemical pathways, diseases, imaging, etc.) at scale has resulted in an enormous amount of human knowledge scattered across data silos. Querying all that knowledge has become remarkably complex. At NExTNet, we use our proprietary Large Language Models stack to mine for associations between scientific content in text (e.g., published literature, clinical trial record, patents) and other public and proprietary databases (gene sequencing, protein expression, diseases, pathways, pathogens, drugs, imaging...) and semantically link them into the world's fastest-growing scientific semantic web. The technical barrier to query and access such scattered knowledge is scarily high: an average researcher would need to have extensive knowledge of the command-line interface by mastering myriads of R libraries, learning a whole suite of Python packages or architecting complex queries using languages such as SQL, SPARQL, etc. NExTNet's cloud-native platform allows these researchers and scientists to ask and answer complex questions in simple natural language without having to master coding, querying languages, or arcane statistics. Our commercial and technological breakthrough is enabling scientists to identify patterns hidden deep within disparate and multi-modal datasets, ranging from scientific papers and clinical trial reports to gene sequencing and protein expression atlases -- so that they can quickly find leads buried in mountains of information via our Graphical User Interface (GUI). We are disrupting the intersection of Language and Generative Artificial Intelligence and the Biomedical industry. Standing Out from The Crowd Knowledge discovery for life sciences R&D is an embryonically developing market. The key problem in biomedicine today: Biomedical knowledge is growing at an exponential rate. As the amount of disparate data (e.g., scientific text, molecular data from sequencing, gene editing and other experiments etc.) increases, it becomes harder and harder for researchers to stay up-to-date with the state of the art. Since the technical barrier to access knowledge buried deep within all these multi-modal data silos is very challenging without the knowledge of the command-line interface, decisions are made without key insights and missed knowledge. At NExTNet, our 10x value increase over competitors is our state-of-the-art Generative AI pipeline not only reads 10s of millions of scientific text (full-length peer-reviewed publications, abstracts, patents, grants etc.) but also analyzes a massive corpus of molecular databases (genes, proteins, and pathways) and mines for hidden associations and patterns across these disparate data (public domain and proprietary). Our competitors are just limited to text mining without expanding into other modalities of data. Our Predictions/Thoughts on the Drug Discovery and Development/ Biomedical Research Industry in 2023 As a startup, we are laser-focusing on the drug discovery and development market due to the heavy $$$ being spent before any clinical trial is done in the pre-clinical phases over the course of 3-7 years at least. There is also a significant amount of VC money being invested in AI drug discovery companies (Atomwise, Insilico, Benevolent AI, Exscientia, Insitro, Recursion, etc. that have their own therapeutic pipeline). Given the significant risks associated with bringing a drug to market coupled with long development timelines (that can stretch up to a decade or even more), engaging with regulatory agencies such as the FDA, etc., and long cycle time of co-development partnerships (e.g. upfronts + milestones + royalties), we decided early on to just focus on the software development and rather enable other biotech and pharma companies developing cutting-edge therapeutics their proprietary data on top of NExTNet's semantic knowledge network --> ask and answer complex biological questions to advance their drug development efforts. A company such as Benchling has found significant success so far as an Enterprise software company. 'contextualize' To give a sense of the massive cost of drug discovery and development: 30 years ago, it took ~$300M USD to bring a drug to market v/s today it takes ~$2.4B USD (accounting for inflation). This is not because any single stage of the discovery and development process for a given molecule has gotten that much more expensive. In fact, there have been meaningful increases in productivity for a number of these stages -- so if anything the total cost of bringing a drug to market should have gotten cheaper. The real reason for such high costs in bringing a drug to market is that most of the drugs that we put in every single phase fail and the funnel has gotten progressively wider and wider narrowing down to a smaller and smaller number of drugs that make it to the next phase. This is clearly a failure of our ability to look at say characteristics (e.g., understanding molecular mechanisms of action, biological processes, etc.) at one stage and predict what's going to succeed in the next stage. This is where better predictive ML models (e.g. natural language AI) can help tremendously. In addition, diseases have traditionally been diagnosed by their symptoms and their location in the body and not by the underlying molecular mechanisms, pathways, or biological processes unique to a specific patient. Identifying targets and discovering connections with pathways early on in the disease progression helps with intervening at the causal stage rather than the symptoms. Hence a lot of the prescribed medicines don't often work on patients. What word defines the state of the ‘ Industry’ in 2023? intersection of AI and Biomedical Language AI Why we decided to participate in HackerNoon's Startup of the Year awards We got invited! :) Final Thoughts Biomedical knowledge is immense and growing quickly. The publicly available knowledge is disconnected - it is impossible to identify patterns/associations hidden deep within such disconnected knowledge. A company's internal knowledge is disconnected from it too. This presents a significant challenge for scientists in the biomedical industry because they: Miss important and potentially relevant knowledge Unable to stay up to date or keep up with new knowledge Decisions are made without the advantage of complete insights At NExTNet, we are building the best-in-class Generative/Language AI stack to enable scientists to identify patterns hidden deep within disparate and multi-modal datasets, ranging from scientific papers and clinical trial reports to gene sequencing and protein expression atlases -- so that they can quickly find leads buried in mountains of information. Start your biomedical knowledge discovery journey by signing up for free right . here is nominated as NExTNet Inc. Startup of the Year in San Francisco, USA. Vote for us today!

Generative AI & Biotech's Future - Interview with Startups of the Year Nominee, NExtNet Inc.

La IA generativa y el futuro de la biotecnología: entrevista con la nominada a Startups del año, NExtNet Inc.

L'avenir de l'IA générative et de la biotechnologie - Entretien avec le candidat de la startup de l'année, NExtNet Inc.

जेनरेटिव एआई और बायोटेक का भविष्य - वर्ष के नामांकित व्यक्ति, नेक्स्टनेट इंक. के स्टार्टअप के साथ साक्षात्कार।

ジェネレーティブ AI とバイオテクノロジーの未来 - Startups of the Year ノミネート、NExtNet Inc. へのインタビュー

IA generativa e futuro da biotecnologia - Entrevista com Startups indicadas do ano, NExtNet Inc.

Tương lai của AI & Công nghệ sinh học sáng tạo - Phỏng vấn Người được đề cử Công ty khởi nghiệp của năm, NExtNet Inc.

生成式人工智能和生物技术的未来 - 采访年度初创公司提名者 NExtNet Inc.

We are very excited to announce that we have raised a new round of funding led by (a16z) with participation from and existing investors. Andreessen Horowitz Y Combinator Continuity This fundraise comes at the back of our impressive growth, as we are now sending for some of the most sophisticated engineering teams at companies both large (Fortune 500) and small (startups). billions of webhooks a year Companies like , , , , and , all use Svix to power their webhooks. Brex Lob Benchling Bizzabo LTSE Equity We have gotten to know the a16z and YC Continuity teams well over the last few months, and we are excited to be working with them for years to come, as well as continuing working with our amazing existing investors: , , Andrew Miklas, Yuri Sagalov ( ), Kevin Mahaffey ( ), Christopher Golda ( ), Jason Warner, Ian Storm Taylor, Kurt Mackey, Hana Mohan, Holly Dunlap, and Michael Orland. Aleph Y Combinator Wayfinder SNR Rogue Capital Today is an important milestone towards our mission of making server-to-server communication easy, secure, and reliable. So I want to take this opportunity to thank the team for getting us to where we are today. I love waking up every day and being surrounded by such smart and passionate individuals, serving amazing customers, and working with the best investors out there! What is Svix? To talk about Svix, we first need to talk about webhooks. Webhooks are how services communicate; you can think of them as sort of a reverse API, or as the API equivalent of push notifications. They power most of the products that we all use on a daily basis. For example: when you make a payment online, a webhook is sent from the payment provider to the merchant; when you push a commit to Github, a webhook is sent to the CI providers to trigger a build; and when you use a workflow automation tool like Zapier, the workflows are almost always triggered by webhooks. Webhooks have also become an increasingly common customer demand, both by API providers and more importantly API consumers. We see this trend across all industries, where customers expect webhooks in order to be able to take action on events without having to poll for updates. It makes sense, as webhooks enable building workflows on top of products, turning them from useful into critical infrastructure, increasing both stickiness and usefulness. The problem, however, is that building a reliable webhook service requires a lot more engineering time, resources, and ongoing maintenance than you first expect. Which is especially painful, as it diverts valuable resources from improving the product, to reinventing infrastructure. We believe webhooks will follow a similar development pattern to other internet infrastructure—like email, web servers, and CMS—which only a few teams build on their own. That's where Svix comes in. With Svix, companies can build a secure, reliable, and scalable webhooks solution in minutes. Saving them a lot of time, effort, and money, while focusing on their business and increasing team happiness. Looking back Milestones are always a great time to reflect on the past. Looking back, I'm really proud of the product we have built, and how we've been supporting our customers. It never gets old to hear "wow, that was really fast" when we release a new feature shortly after discussing it with a customer. We deserve some credit for that (and we work hard to earn it), but a lot of the credit also belongs to our customers. We are very lucky to have such engaged customers, and doubly lucky to have them be developers. We often get very detailed feedback, specced out feature requests, and of course pull requests for both the code and the docs. I think this has also been the secret to why our customers love the product so much. It's because they helped us build it every step of the way, and they still do to this day. The same way it never gets old to hear about how fast we move with product development, it also never gets old to hear about how people use the product and how much they love it. This is by far one of the best products I've ever worked with. It does one thing but it does it
so well that I can't imagine ever doing my own webhooks solution. - Senior Engineering Manager at Bizzabo One super important but often underappreciated aspect of what we do, is the relentless focus on stability and scalability. We spend a lot of time and work really hard to ensure that we provide our customers with a robust service at the scale they need. Whether it's the quality of the code, the test suite, the infrastructure, or our SDLC, we always hold ourselves to the highest standards. It's one of those things that no one notices when you do your job well, but are essential for any infrastructure service. At the end of the day, however, it's not about what we have built, but rather how we are affecting the lives of our customers. Thanks to Svix our customers can offer a state of the art webhooks solution to their customers at a fraction of the time and cost it would take to build it themselves, with no ongoing maintenance, while getting constant improvements and automatic scaling. All of this while freeing their engineering teams to work on their core business. For more on that, you can see what . Lob had to say about their experience switching to Svix Looking forward We have identified an important (and growing) problem, built a great product to solve it, and started a whole new vertical along the way (webhooks as a service). This is why our customers, our investors, and the Svix team are so excited about webhooks and what we are building. We have an opportunity to become an industry defining company, which is why we have decided to raise this round of funding, and partner with some of the best investors out there. We are going to use the money and the support of our new investors, to further expand the team and improve the product to keep up with the demand and our ever evolving customer needs.  We have a lot of exciting stuff lined up, and we can't wait to share them! If you are passionate about developer tools, webhooks, and server-to-server communication and you think you may be a good fit, please check out our . careers page If your company is looking to start sending webhooks, or upgrade their existing legacy solution, or . try us out check out our docs Also published here.

How My Startup Got New Funding from Andreessen Horowitz

Cómo mi startup obtuvo nuevos fondos de Andreessen Horowitz

Comment ma startup a obtenu un nouveau financement d'Andreessen Horowitz

कैसे मेरे स्टार्टअप को आंद्रेसेन होरोविट्ज़ से नई फंडिंग मिली

私のスタートアップがアンドリーセン・ホロウィッツから新しい資金を調達した方法

Como minha startup conseguiu novos financiamentos da Andreessen Horowitz

Làm thế nào Startup của tôi có được nguồn vốn mới từ Andreessen Horowitz

我的初创公司如何从 Andreessen Horowitz 获得新资金

What is serverless? Serverless is a cloud computing model that was first introduced by AWS in 2014 (AWS Lambda is the market leader to this day). It abstracts away most of the server operations to the cloud provider so developers can only focus on writing code and shipping new features. Serverless adoption is rapidly growing and it's mostly due to its promise of providing significant cost and time efficiency for technology companies. For new startups, just starting to build their first product, it's a no-brainer to build it on serverless. It's very cheap and quick to get started. You don't have to have that much knowledge about the backend processes, since the cloud providers handle all that and you can just focus on building your product and it's functionality. The go-to-market time is much shorter than with previous computing models and it scales automatically when you get successful and have tons of users pouring on your site. See, a no-brainer! With already established companies, it's harder to switch to serverless because of the legacy systems and processes that they have in place, but I'm here to argue that it might be worth the pain! Big companies = big savings For established companies (depending on their specific use case), serverless architecture gives the opportunity to save money that was previously paid for hosting idle code. With serverless, you only pay for it when your code is executed. The bigger the company and the more complex its technology has -- the bigger the potential savings can be. FINRA's experience with cost reduction: "Using AWS Lambda, we've increased cost efficiency by a factor of two. We only pay for what we use, and we don't have to manage on-premises server infrastructure." For new startups these costs can be trivial, but for some companies it can be a make-or-break question to get these costs down. For example, some companies have reported just by switching to serverless. close to 90% of cost reductions A promise of reliability and availability to the users Slightly bigger companies usually already have an established user base that is also rapidly growing. It means that they have the One of the nastiest things that can happen (and very often occurs in growing technology companies) is that your service gets a sudden peak in traffic -- either it's a really successful marketing campaign that just launched or some other reason that spikes the usage of your product very suddenly -- and your site can't handle that extra load. As a result, your customers get a bad experience with your product at the most unfortunate time -- when you are finally hitting it big. responsibility to keep the service reliable at all times. Why serverless is so useful -- instead of constantly being ready for the potential spike and paying for the extra servers, you can let Now the bigger influx of users won't just be a happy moment for your marketing team, but the engineers can also sleep well at night. cloud providers worry about automatic scaling. Square Enix experience with automatic scaling: At a New Year's Eve event after migration to AWS Lambda, screenshots were taken at a rate of more than 6,000 per minute. "AWS Lambda had a striking effect. Image processing that used to take several hours was finished in a little over 10 seconds." Saving money and time because less server ops is needed In established companies that have been around for some years, you might have lots of people working only on provisioning and scaling servers. Once this is "outsourced" to cloud providers, these people can restructure their work and focus on more business-critical tasks. Depending on the size of the company and the structure of the development team, it can be a huge benefit that not only saves time and money, but can also make hiring for some positions easier. Alt/S experience with reducing server ops: Zero time is now spent deploying or maintaining server instances, as Sportify's entire backend platform is based on AWS Lambda. The technical team will focus on the business logic Serverless computing shortens the time from code development to putting that code in production. This puts the focus of the development team strictly on writing code that benefits the users or follows some specific business logic and goals. Since serverless is just another layer of abstraction in cloud computing, its main benefit is to allow dev teams to write and ship better code, better functions, and therefore better products. It's the rocket fuel for future innovations and bigger companies can't afford to stop innovating. Benchling experience with better business focus: Instead of worrying about the solution architecture and managing servers, they are freed up to focus on new projects and initiatives that can help grow the company. Increased agility and more efficient dev teams It's commonly known that as companies grow, the execution speed goes down and there are more processes and more bureaucracy involved with every task. Since it's very easy and quick to launch new functionality (and also to scrap it) with serverless, established companies can reverse this process and bring back the experimental feature launching speed that small startups enjoy. Autodesk experience with increased agility: Building our product (Tailor) on a serverless architecture made it easy to modularize it for easier sharing with other divisions, or as an open-source package with non-Autodesk entities that have a use for it. Why some companies are still hesitating with the switch? Serverless technology is not in its infancy anymore and yet it's not fully mature either. ...At least that's what the hesitant-to-jump-on-new-technology big companies are saying. Yaron Haviv, founder and CTO of iguaz.io, says: "The interesting part is that by abstracting away much of the Docker/Kubernetes complexity, serverless can be adopted faster and more easily, even though it's the newer tech." There are also some unique challenges that come with this new computing paradigm -- slow cold starts, short-lived functions, a closed set of triggers, and a lack of observability and debugging options. There are workarounds and viable solutions for solving these though thanks to a growing community that works on documenting use cases and best practices. One of the biggest problems -- lack of observability for the distributed serverless systems -- is solved by several newly emerged third-party tools, the most notable one being Dashbird. It gathers CloudWatch logs along with X-Ray and API Gateway data and gives a general overview of all your functions -- along with error alerts, tracing, live tailing, and much more. Conclusion Despite being careful about jumping on another bandwagon too quickly, established companies have a lot to win from migrating to serverless. Since serverless adoption is already skyrocketing, big businesses should not wait too long to get on board, because this new paradigm shift favors quick shipping and constant innovation -- and that's exactly what might give some new and hungry startups the edge to go against dinosaurs and beat them? Also published here: //dashbird.io/blog/serverless-business-benefits-for-big-companies/

How Migrating to Serverless Gave Square Enix, Autodesk, and FINRA Big Business Benefits

Software as a Service (SaaS) is an elegant model. Once you build the product, you can sell it to any number of customers. The only changes required are minor tweaks to the configuration. SaaS businesses work by charging a recurring fee. At each billing cycle, we need to re-earn the trust of our customers, creating a powerful alignment of interest between us and the customers we serve.  For customers, SaaS just works™. They don’t have to install anything, and paying a smaller recurring fee is a lot more palatable on the balance books. For us, has two benefits. First, having recurring revenue makes our finances predictable. Second, it cuts down development overhead, since new products can be rolled out simultaneously to all clients. To understand the SaaS business, let’s deconstruct the levers that control it. business SaaS Financially, the objective of a business is to increase profitability, where: Profit = Revenue — Cost There are two ways we can increase profitability. Either we increase revenue or decrease costs. Revenue For a SaaS business, revenue can be modeled as: Revenue = Number of customers * Revenue/Customer or more completely, Revenue = Number of customers * Number of product types * Price per product type * Number of seats/customer * Retention rate To increase revenue, we control the following levers: Number of product types sold There are two ways to increase the number of product types we sell to customers: The more impactful but costly approach is to develop new product lines to sell. For example, at Benchling, we built new product lines such as Workflows and Requests, which expanded the set of products available for purchase. Similarly, Atlassian lets you buy Jira and Confluence as separate products. Build more products. Create tiers that appeal to different segments of the market by gating functionality. For example, at Benchling, we can charge a premium on Notebook Plus over our basic Notebook product to target teams collecting structured data. Selling a read-only license enables us to tap into seats that clients would typically shy away from purchasing. Segment existing product lines. Price per product type The easiest way to grow revenue is to raise the price of each product line. As long as the market is willing to pay more, growing revenue by raising prices requires no development. In an ideal world, we have perfect price inelasticity, i.e. we can raise the price without losing any customers. That will only happen when our product offers unique value that no-one else can match. To figure out just how high we can price the product, we triangulate pricing for SaaS through a few strategies. Understand what value we are adding, and capture some percentage of that value. Value-based pricing. Look at what others are charging for similar products. Competitive pricing. Experiment with raising the price to measure its impact on demand. Experimental pricing. Number of seats per customer A non-exhaustive set of strategies to increase the number of seats: Give your champions tools to convince others. Offer training sessions, support, access to help articles, and special features. Empower evangelists. Most companies worry about people using their software without paying for it. The far bigger challenge in enterprise software is getting people to use your product in the first place. Instead of gating access to features, enable them by default and monitor usage. If people start using features they didn’t pay for, after a few cycles bring up extending the contract or offering to take them away after. If people have already spent time training themselves and collecting data against it, it’s unlikely that they will back away from paying the upgrade fee. Default to permissiveness. People rarely work with just one tool. Build the bridges that let them collaborate with other products by either supporting integrations or easier ways to share out from the platform. For example, Slack Apps integrate with an extensive suite of business apps that enhance its utility. Support extensibility. Retention Retention is a measure of the number of completed payment cycles. Customers churn when someone else does a significantly better job of delivering value. Competitors need to create such a compelling proposition that it persuades our customers to overcome inertia, training, and migration costs. To lose a customer means that we weren’t listening.  To ensure customers stay, SaaS businesses need a Customer Success (CS) team. The tactics used by CS to drive retention are: Teaching not just the capabilities of the software, but also the workarounds. Training users. Ensuring that people know how to get timely help if they get stuck. Being responsive. As the first group to hear the customer pain points, CS is in a unique position to share feedback with the product team on what people are trying to do, their difficulties, and the severity of the issue. Sharing feedback with the product team. Number of customers The number of customers can be modeled as Number of customers = Acquisitions * Conversion rate The levers we have are either increasing the number of leads coming in through the gate, or increasing our success at converting them into paying customers. While the strategies for growing the number of customers is too broad a topic to cover here, we have marketing and sales organizations that are uniquely focused on this. Costs Key costs in a SaaS business can broadly be identified as: Payroll costs for engineers, designers, and other members of the product development team. For SaaS businesses, this tends to be one of the largest expenses. For example, a team of 10 engineers working on a product for a year can easily require $1.2 million. Development: Time invested in training and supporting customers. Support: Costs such as servers and storage that need to be paid to AWS for running the business. Operations: Fees paid to vendors such as credit card processors for payments, etc. Transactions: Financial model The core thesis of a SaaS business is that the costs grow roughly linearly, whereas revenue grows exponentially. To see it in action, let’s create a very simple business model: Where revenue is: Revenue = Number of product lines* Avg price per product line* Number of customers * (1 + customer growth) ^ n* Avg # of seats/customer * (1 + upsell rate) ^ n- Previous payment cycle revenue * Churn rate and cost is Cost = (Operational cost of deploy * Number of customers)+ (Avg annual salary * (Number of engineers + New engineers/year * n))- (Transaction fee * Gross Revenue) While we start in the red, over a period of time we see incredibly attractive returns. Plotting the cumulative values against each other, over time we get: Understanding the levers at play empowers us to diagnose issues and drive outcomes. What levers have you found impactful in managing your SaaS business?

Understanding the business of SaaS

Trust Your Gut—Because Sometimes Your Brain is Full of Sh*t

When just starting out in web design and development, Alfonso de la Nuez imagined a better way to build websites more focused on the people using them instead of the web designers (or CEO of the companies) creating them. When beginning his career in the late 90s, the phrase “user experience” (UX) was not yet seared into the tech lexicon. Alfonso’s efforts to prioritize the user over the next decades would prove to be pivotal in changing the SaaS landscape. Alfonso brings lessons of selflessness gained from a lifetime of playing basketball at a high level to the company he co-founded: UserZoom, which makes robust user experience (UX) research accessible to anyone savvy enough to have it as a cornerstone of product development and marketing. Whether playing on a Division I basketball squad fighting to make the playoffs or running a startup technology company reorganizing personnel as it scales, Co-founder and Chief Visionary Officer (CVO) has been a part of many tough decisions to call on the right talent at the right time. Sometimes this included benching himself when he thought someone else could do a better job. UserZoom Alfonso de la Nuez A staunch advocate of putting team mission before personal stats, Alfonso is also a firm believer in falling in love with the problem you are trying to solve before launching your business. Such a focused mindset can be crucial in gauging the market for your ideas before getting too lost in other distractions. In addition to identifying UX as a fundamental consideration for any SaaS company building new products, Alfonso also noticed a gap early on between companies’ efforts to prioritize UX, and the costly methods and resources required to effectively collect actionable data about user preferences and behavior. UserZoom helps companies close that gap, presenting insights about UX in an affordable, efficient manner that reveals information and findings from user interactions with prototypes or live software . in hours Alfonso spoke with me on an episode of the about his victories and missteps over the years, and generously shares critical lessons from his experience, which he also outlines in his new book “ .” SaaS Origin Stories podcast The Digital Experience Company: Winning in the Digital Economy A Two-Way Mirror to Reflect Reality Leading up to the dot-com collapse in the early 2000s, many new companies were scoring high valuations, based on management teams’ visions that did not incorporate into business models the actual interactions individuals had with websites and products. Efforts to test and measure the real-world experiences users were having with digital products were just taking shape, and techniques to record UX interactions started as just that, recordings. UX consulting company teams, like the one Alfonso worked with, captured footage with video cameras as test subjects navigated websites or tried out software products his clients were designing or researching. The subjects were situated in physical labs behind a two-way mirror and instructed to think out loud as they performed tasks designed to assess usability. Alfonso and his team described themselves as “user advocates” and invited their clients to observe the tests in person from the other side of the mirror. As his customers watched users struggle, Alfonso shares that, “a lot of times, it was funny. Other times, it was more dramatic. The customer would observe from the other side of the room and say, ‘…just click there! Don’t you see? It’s right there! Click there!’” The client reactions to users failing miserably to utilize their software were so priceless that they ended up prominently featured in marketing efforts to demonstrate the value of such research. Currently, automation and remote accessibility are key selling points of UX research software like UserZoom — features that don’t require quite as much staging. Full Service or DIY Subscription? After spending years refining UX research, Alfonso helped to launch UserZoom in 2007, focusing on providing UX insights that were just as revealing as in-person tests but used software to provide data more quickly and efficiently. UserZoom began by offering a service-based model, wherein companies would hire the organization to do the work for them. High-touch, consultative sales efforts touted the value of delegating research and measurement efforts out-of-house and gave companies a team of experts ready to work on their behalf. Initially focusing on sophisticated, advanced clients, the Spanish company secured large contracts with technology juggernauts like Google, which recognized the importance of placing the user at the forefront of product development. Two years later, around 2009, Alfonso describes having conversations with the tech giant about a different model based on licenses (not services). “Google told us … you guys are great guys, and you provide a great service. [Now] we want the keys to the car.” As demand for a subscription model grew, UserZoom pivoted to their product, giving customers usernames and passwords to access the UX insight software on their own. The services-based approach fell entirely to the wayside for a period of time and, eventually, a hybrid model offering both services and subscriptions prevailed, essentially creating two very different branches of the company. “It was hard [basically] running two companies at once,” Alfonso admits. Given the chance to go back, he would hire more people and raise more capital to keep both models in play for the duration of the growth process and allocate more dedicated resources to each approach. Keeping Some Skin in the Game Alfonso (originally from Madrid) initially endured significant challenges and hardships helping to build the Spanish company from the ground up on U.S. soil. As UserZoom continued growing, it became apparent that leveraging capital from outside sources could be a wise strategy to maximize the company’s potential. Networking in places such as the in Silicon Valley, Alfonso’s team put considerable time into establishing connections in the industry. That effort paid off — they did not have to solicit investors at all when it came time to raise capital. Plug and Play Tech Center These contacts could see the value in their product, as well as the patience and strategy the company’s leaders had demonstrated in assessing the market. Alfonso makes an important distinction between the choices he and his colleagues faced when it came time to compare offers from private equity (PE) and venture capital (VC) firms. PE firms tend to prefer a majority investment, with plenty of control over operations and significant interest in the steady, profitable growth of the company in which they have a stake. VC firms can have a much different approach, with greater emphasis on minority investment positions, hyper growth at all costs, and rapidly increasing market share. In addition to assessing the culture, mission, and goals of PE and VC, Alfonso recommends discussing motives and strategic plans upfront with investors and stepping out of the way at different stages of growth. Alfonso points to his ability to put ego aside as one of the most important factors explaining his current Chief Visionary Officer role at UserZoom. New leadership decided to keep him on board once PE came into the picture, in contrast to the fate of many other executives after a large acquisition: termination. As he puts it, stepping aside into a different role is “very hard to do when you’ve been the founder and you’ve created a business, and you think that [you’re] Michael Jordan … or Spider-Man, or some sort of superhero — which you are, by the way, because it’s very difficult to do that.” There’s No Excuse Not To Have a Phenomenal UX Now that accessible tools exist to gather actionable data fast, Alfonso believes that all SaaS entrepreneurs and leaders would do well to appreciate the competitive advantage that an easy, memorable user experience can provide. “The user becomes your greatest salesperson,” he explains. Product-led growth (PLG) is a core strategy of many SaaS companies in the current market, and pioneers searching for advice as they build new products can look to the three mistakes Alfonso openly discusses as cautionary tales from his journey to success: Employ a hybrid sales model combining subscriptions and services. Looking back, Alfonso thinks that his team missed opportunities by fully dropping the service model for so long. Invest earlier in UX design. “I’d hire a product designer and a product leader before I’d hire a salesperson,” Alfonso says. Raise capital earlier. Bootstrapping operations can mean saving critical funds and developing a scrappy, resourceful company culture that can yield dividends in many ways. However, suffering doesn’t inherently breed success, and leveraging outside capital, in a measured and responsible way, can accelerate wins in a uniquely exponential fashion. Alfonso wishes he had realized that sooner. Alfonso suggests that humility also played a role in his longevity, and attributes much of his continued accomplishments to the wisdom in asking for help. As he says, “How many people can raise their hand and say, ‘I’m not the right person to make this decision,’ or simply, ‘I don’t know.’” From having difficult conversations about reorganization to deciding whether to loosen your grip on the reins, Alfonso’s path has not been , but has been made by understanding the importance of evolving throughout changes and putting the interests of the organization and consumer above your own immediate impulses. easy easier This article is based on an episode of the podcast, hosted by , a digital product development consultancy. Subscribe via , or anywhere you get podcasts to hear first-hand do’s and don’ts of building and growing a SaaS, as well as inspirational stories to fuel you on your own journey into the world of SaaS. SaaS Origin Stories DevSquad Apple , Spotify

It’s Not About You: Finding Success Through Managing Ego and Focusing on End Users

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix 3 METHOD The goal of this work is to align an open-source large-language model to the intent of the user. Throughout the work we assume access to a larger teacher model πT which can be queried by prompted generation. Our goal is be to produce a student model πθ and our approach follows similar stages as InstructGPT (Ouyang et al., 2022) as shown in Figure 2. Distilled Supervised Fine-Tuning (dSFT) Starting with a raw LLM, we first need to train it to respond to user prompts. This step is traditionally done through supervised fine tuning (SFT) on a dataset of high-quality instructions and responses (Chung et al., 2022; Sanh et al., 2021). Given access to a teacher language models, we can instead have the model generate instructions and responses (Taori et al., 2023), and train the model directly on these. We refer to this as distilled SFT (dSFT). Approaches to dSFT follow the self-instruct protocol (Wang et al., 2023). Let x 0 1 , . . . , x0 J be a set of seed prompts, constructed to represent a diverse set of topical domains. A dataset is constructed through iterative self-prompting where the teacher is used to both respond to an instruction and refine the instruction based on the response. For each x 0 , we first sample response y 0 ∼ πT(·|x 0 ), and then refine by sampling a new instruction (using a prompt for refinement), x 1 ∼ πT(·|x 0 , y0 ). The end point is a final dataset, C = {(x1, y1), . . . ,(xJ , yJ )}. Distillation is performed by SFT, AI Feedback through Preferences (AIF) Human feedback (HF) can provide additional signal to align LLMs. Human feedback is typically given through preferences on the quality of LLM responses (Ouyang et al., 2022). For distillation, we instead use AI preferences from the teacher model on generated outputs from other models. We follow the approach of UltraFeedback (Cui et al., 2023) which uses the teacher to provide preferences on model outputs. As with SFT, the system starts with a set of prompts x1, . . . , xJ . Each prompt x is fed to a collection of four models π1, . . . , π4, e.g. Claude, Falcon, Llama, etc, each of which yield a response y 1 ∼ π1(·|x), . . . , y4 ∼ π4(·|x). These responses are then fed to the teacher model, e.g. GPT-4, which gives a score for the response s 1 ∼ πT (·|x, y1 ), . . . , s4 ∼ πT (·|x, y4 ). After collecting the scores for a prompt x, we save the highest scoring response as yw and a random lower scoring prompt as yl . The final feedback dataset D consists of a set of these triples (x, yw, yl). Distilled Direct Preference Optimization (dDPO) The goal of the final step is to refine the πdSFT by maximizing the likelihood of ranking the preferred yw over yl in a preference model. The preference model is determined by a reward function rθ(x, y) which utilizes the student language model πθ. Past work using AI feedback has primarily focused on using RL methods such as proximal policy optimization (PPO) to optimize θ with respect to this reward. These approaches optimize θ by first training the reward and then sampling from the current policy to compute updates. Direct preference optimization (DPO) uses a simpler approach to directly optimize the preference model from the static data (Rafailov et al., 2023). The key observation is to derive the optimal reward function in terms of the optimal LLM policy π∗ and the original LLM policy πdSFT. Under an appropriate choice of preference model they show, for constant β and partition function Z that, By plugging this function of the reward into the preference model, the authors show that the objective can be written as, While this term looks complex, we note that it implies a simple training procedure. Starting with the dSFT version of the model, we iterate through each AIF triple (x, yw, yl). 1. Compute the probability for (x, yw) and (x, yl) from the dSFT model (forward-only). 2. Compute the probability for (x, yw) and (x, yl) from the dDPO model. 3. Compute Eq 1 and backpropagate to update. Repeat. This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix 3 METHOD The goal of this work is to align an open-source large-language model to the intent of the user. Throughout the work we assume access to a larger teacher model πT which can be queried by prompted generation. Our goal is be to produce a student model πθ and our approach follows similar stages as InstructGPT (Ouyang et al., 2022) as shown in Figure 2. Distilled Supervised Fine-Tuning (dSFT) Starting with a raw LLM, we first need to train it to respond to user prompts. This step is traditionally done through supervised fine tuning (SFT) on a dataset of high-quality instructions and responses (Chung et al., 2022; Sanh et al., 2021). Given access to a teacher language models, we can instead have the model generate instructions and responses (Taori et al., 2023), and train the model directly on these. We refer to this as distilled SFT (dSFT). Distilled Supervised Fine-Tuning (dSFT) Approaches to dSFT follow the self-instruct protocol (Wang et al., 2023). Let x 0 1 , . . . , x0 J be a set of seed prompts, constructed to represent a diverse set of topical domains. A dataset is constructed through iterative self-prompting where the teacher is used to both respond to an instruction and refine the instruction based on the response. For each x 0 , we first sample response y 0 ∼ πT(·|x 0 ), and then refine by sampling a new instruction (using a prompt for refinement), x 1 ∼ πT(·|x 0 , y0 ). The end point is a final dataset, C = {(x1, y1), . . . ,(xJ , yJ )}. Distillation is performed by SFT, AI Feedback through Preferences (AIF) Human feedback (HF) can provide additional signal to align LLMs. Human feedback is typically given through preferences on the quality of LLM responses (Ouyang et al., 2022). For distillation, we instead use AI preferences from the teacher model on generated outputs from other models. AI Feedback through Preferences (AIF) We follow the approach of UltraFeedback (Cui et al., 2023) which uses the teacher to provide preferences on model outputs. As with SFT, the system starts with a set of prompts x1, . . . , xJ . Each prompt x is fed to a collection of four models π1, . . . , π4, e.g. Claude, Falcon, Llama, etc, each of which yield a response y 1 ∼ π1(·|x), . . . , y4 ∼ π4(·|x). These responses are then fed to the teacher model, e.g. GPT-4, which gives a score for the response s 1 ∼ πT (·|x, y1 ), . . . , s4 ∼ πT (·|x, y4 ). After collecting the scores for a prompt x, we save the highest scoring response as yw and a random lower scoring prompt as yl . The final feedback dataset D consists of a set of these triples (x, yw, yl). Distilled Direct Preference Optimization (dDPO) The goal of the final step is to refine the πdSFT by maximizing the likelihood of ranking the preferred yw over yl in a preference model. The preference model is determined by a reward function rθ(x, y) which utilizes the student language model πθ. Past work using AI feedback has primarily focused on using RL methods such as proximal policy optimization (PPO) to optimize θ with respect to this reward. These approaches optimize θ by first training the reward and then sampling from the current policy to compute updates. Distilled Direct Preference Optimization (dDPO) Direct preference optimization (DPO) uses a simpler approach to directly optimize the preference model from the static data (Rafailov et al., 2023). The key observation is to derive the optimal reward function in terms of the optimal LLM policy π∗ and the original LLM policy πdSFT. Under an appropriate choice of preference model they show, for constant β and partition function Z that, By plugging this function of the reward into the preference model, the authors show that the objective can be written as, While this term looks complex, we note that it implies a simple training procedure. Starting with the dSFT version of the model, we iterate through each AIF triple (x, yw, yl). 1. Compute the probability for (x, yw) and (x, yl) from the dSFT model (forward-only). 2. Compute the probability for (x, yw) and (x, yl) from the dDPO model. 3. Compute Eq 1 and backpropagate to update. Repeat. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Method

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix ABSTRACT We aim to produce a smaller language model that is aligned to user intent. Previous research has shown that applying distilled supervised fine-tuning (dSFT) on larger models significantly improves task accuracy; however, these models are unaligned, i.e. they do not respond well to natural prompts. To distill this property, we experiment with the use of preference data from AI Feedback (AIF). Starting from a dataset of outputs ranked by a teacher model, we apply distilled direct preference optimization (dDPO) to learn a chat model with significantly improved intent alignment. The approach requires only a few hours of training without any additional sampling during fine-tuning. The final result, ZEPHYR7B, sets a new state-of-the-art on chat benchmarks for 7B parameter models, and requires no human annotation. In particular, results on MT-Bench show that ZEPHYR-7B surpasses LLAMA2-CHAT-70B, the best open-access RLHFbased model. Code, models, data, and tutorials for the system are available at //github.com/huggingface/alignment-handbook. 1 INTRODUCTION Smaller, open large language models (LLMs) have greatly increased in ability in recent years, from early GPT-2-like models (Wang & Komatsuzaki, 2021) to accurate and compact models (Touvron et al., 2023; Penedo et al., 2023; Jiang et al., 2023) that are trained on significantly more tokens than the “compute-optimal” amount suggested by the Chincilla scaling laws (De Vries, 2023). In addition, researchers have shown that these models can be further trained through distilled supervised fine-tuning (dSFT) based on proprietary models to increase their accuracy (Taori et al., 2023). In this approach, the output of a more capable teacher model is used as supervised data for the student model. Distillation has proven to be an effective tool for improving open models on a range of different tasks (Chiang et al., 2023); however, it does not reach the performance of the teacher models (Gudibande et al., 2023). Users have noted that these models are not “intent aligned”, i.e. they do not behave in a manner that aligns with human users’ preferences. This property often leads to outputs that do not provide correct responses to queries. Intention alignment has been difficult to quantify, but recent work has led to the development of benchmarks like MT-Bench (Zheng et al., 2023) and AlpacaEval (Li et al., 2023) that specifically target this behavior. These benchmarks yield scores that correlate closely with human ratings of model outputs and confirm the qualitative intuition that proprietary models perform better than open models trained with human feedback, which in turn perform better than open models trained with distillation. This motivates careful collection of human feedback for alignment, often at enormous cost at scale, such as in LLAMA2-CHAT (Touvron et al., 2023). In this work, we consider the problem of aligning a small open LLM entirely through distillation. The main step is to utilize AI Feedback (AIF) from an ensemble of teacher models as preference data, and apply distilled direct preference optimization as the learning objective (Rafailov et al., 2023). We refer to this approach as dDPO. Notably, it requires no human annotation and no sampling compared to using other approaches like proximal preference optimization (PPO) (Schulman et al., 2017). Moreover, by utilizing a small base LM, the resulting chat model can be trained in a matter of hours on 16 A100s (80GB). To validate this approach, we construct ZEPHYR-7B, an aligned version of Mistral-7B (Jiang et al., 2023). We first use dSFT, based on the UltraChat (Ding et al., 2023) dataset. Next we use the AI feedback data collected in the UltraFeedback dataset (Cui et al., 2023). Finally, we apply dDPO based on this feedback data. Experiments show that this 7B parameter model can achieve performance comparable to 70B-parameter chat models aligned with human feedback. Results show improvements both in terms of standard academic benchmarks as well as benchmarks that take into account conversational capabilities. Analysis shows that the use of preference learning is critical in achieving these results. Models, code, and instructions are available at //github.com/huggingface/alignment-handbook. We note an important caveat for these results. We are primarily concerned with intent alignment of models for helpfulness. The work does not consider safety considerations of the models, such as whether they produce harmful outputs or provide illegal advice (Bai et al., 2022). As distillation only works with the output of publicly available models this is technically more challenging to do because of added challenges in curating that type of synthetic data, and is an important subject for future work. This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix ABSTRACT We aim to produce a smaller language model that is aligned to user intent. Previous research has shown that applying distilled supervised fine-tuning (dSFT) on larger models significantly improves task accuracy; however, these models are unaligned, i.e. they do not respond well to natural prompts. To distill this property, we experiment with the use of preference data from AI Feedback (AIF). Starting from a dataset of outputs ranked by a teacher model, we apply distilled direct preference optimization (dDPO) to learn a chat model with significantly improved intent alignment. The approach requires only a few hours of training without any additional sampling during fine-tuning. The final result, ZEPHYR7B, sets a new state-of-the-art on chat benchmarks for 7B parameter models, and requires no human annotation. In particular, results on MT-Bench show that ZEPHYR-7B surpasses LLAMA2-CHAT-70B, the best open-access RLHFbased model. Code, models, data, and tutorials for the system are available at //github.com/huggingface/alignment-handbook. 1 INTRODUCTION Smaller, open large language models (LLMs) have greatly increased in ability in recent years, from early GPT-2-like models (Wang & Komatsuzaki, 2021) to accurate and compact models (Touvron et al., 2023; Penedo et al., 2023; Jiang et al., 2023) that are trained on significantly more tokens than the “compute-optimal” amount suggested by the Chincilla scaling laws (De Vries, 2023). In addition, researchers have shown that these models can be further trained through distilled supervised fine-tuning (dSFT) based on proprietary models to increase their accuracy (Taori et al., 2023). In this approach, the output of a more capable teacher model is used as supervised data for the student model. Distillation has proven to be an effective tool for improving open models on a range of different tasks (Chiang et al., 2023); however, it does not reach the performance of the teacher models (Gudibande et al., 2023). Users have noted that these models are not “intent aligned”, i.e. they do not behave in a manner that aligns with human users’ preferences. This property often leads to outputs that do not provide correct responses to queries. Intention alignment has been difficult to quantify, but recent work has led to the development of benchmarks like MT-Bench (Zheng et al., 2023) and AlpacaEval (Li et al., 2023) that specifically target this behavior. These benchmarks yield scores that correlate closely with human ratings of model outputs and confirm the qualitative intuition that proprietary models perform better than open models trained with human feedback, which in turn perform better than open models trained with distillation. This motivates careful collection of human feedback for alignment, often at enormous cost at scale, such as in LLAMA2-CHAT (Touvron et al., 2023). In this work, we consider the problem of aligning a small open LLM entirely through distillation. The main step is to utilize AI Feedback (AIF) from an ensemble of teacher models as preference data, and apply distilled direct preference optimization as the learning objective (Rafailov et al., 2023). We refer to this approach as dDPO. Notably, it requires no human annotation and no sampling compared to using other approaches like proximal preference optimization (PPO) (Schulman et al., 2017). Moreover, by utilizing a small base LM, the resulting chat model can be trained in a matter of hours on 16 A100s (80GB). To validate this approach, we construct ZEPHYR-7B, an aligned version of Mistral-7B (Jiang et al., 2023). We first use dSFT, based on the UltraChat (Ding et al., 2023) dataset. Next we use the AI feedback data collected in the UltraFeedback dataset (Cui et al., 2023). Finally, we apply dDPO based on this feedback data. Experiments show that this 7B parameter model can achieve performance comparable to 70B-parameter chat models aligned with human feedback. Results show improvements both in terms of standard academic benchmarks as well as benchmarks that take into account conversational capabilities. Analysis shows that the use of preference learning is critical in achieving these results. Models, code, and instructions are available at //github.com/huggingface/alignment-handbook. We note an important caveat for these results. We are primarily concerned with intent alignment of models for helpfulness. The work does not consider safety considerations of the models, such as whether they produce harmful outputs or provide illegal advice (Bai et al., 2022). As distillation only works with the output of publicly available models this is technically more challenging to do because of added challenges in curating that type of synthetic data, and is an important subject for future work. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Abstract and Introduction

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix 4 EXPERIMENTAL DETAILS We conduct all of our fine-tuning experiments using Mistral 7B (Jiang et al., 2023), which is the current state-of-the-art base LM at the 7B parameter scale, and matches the performance of much larger models like LLaMa 34B on many NLP benchmarks. We use the Transformer Reinforcement Learning (TRL) library for fine-tuning (von Werra et al., 2020), in conjunction with DeepSpeed ZeRO3 (Rajbhandari et al., 2020) and FlashAttention-2 (Dao, 2023) to optimize memory and improve training speed. All models are trained with the AdamW optimizer and no weight decay. We did not experiment with parameter-efficient techniques such as LoRA (Hu et al., 2021), but expect similar results to hold with these methods. All experiments were run on 16 A100s using bfloat16 precision and typically took 2-4 hours to complete. For the full set of hyperparameters and instructions on how to train the models, see: //github.com/huggingface/alignment-handbook. 4.1 DATASETS We focus on two dialogue datasets that have been distilled from a mix of open and proprietary models, and have previously been shown to produce strong chat models like the UltraLM (Ding et al., 2023): • UltraChat (Ding et al., 2023) is a self-refinement dataset consisting of 1.47M multi-turn dialogues generated by GPT-3.5-TURBO over 30 topics and 20 different types of text material. We initially ran dSFT over the whole corpus, but found the resulting chat model had a tendency to respond with incorrect capitalization and would preface its answers with phrases such as “I don’t have personal experiences”, even for straightforward questions like “How do I clean my car?”. To handle these issues in the training data, we applied truecasing heuristics to fix the grammatical errors (approximately 5% of the dataset), as well as several filters to focus on helpfulness and remove the undesired model responses. The resulting dataset contains approximately 200k examples. • UltraFeedback (Cui et al., 2023) consists of 64k prompts, each of which have four LLM responses that are rated by GPT-4 according to criteria like instruction-following, honesty, and helpfulness. We construct binary preferences from UltraFeedback by selecting the highest mean score as the “chosen” response and one of the remaining three at random as “rejected”. We opted for random selection instead of selecting the lowest-scored response to encourage diversity and make the DPO objective more challenging. As noted above, this step is computed offline and does not involve any sampling from the reference model. We make the pre-processed datasets available on the Hugging Face Hub.1 4.2 EVALUATION Our main evaluations are on single-turn and multi-turn chat benchmarks that measure a model’s ability to follow instructions and respond to challenging prompts across a diverse range of domains: • MT-Bench (Zheng et al., 2023) is a multi-turn benchmark that consists of 160 questions across eight different areas of knowledge. In this benchmark, the model must answer an initial question, and then provide a second response to a predefined followup question. Each model response is then rated by GPT-4 on a scale from 1-10, with the final score given by the mean over the two turns. • AlpacaEval (Li et al., 2023) is a single-turn benchmark where a model must generate a response to 805 questions on different topics, mostly focused on helpfulness. Models are also scored by GPT-4, but the final metric is the pairwise win-rate against a baseline model (text-davinci-003). We also evaluate ZEPHYR-7B on the Open LLM Leaderboard (Beeching et al., 2023), which measures the performance of LMs across four multiclass classification tasks: ARC (Clark et al., 2018), HellaSwag (Zellers et al., 2019), MMLU (Hendrycks et al., 2021), and Truthful QA(Lin et al., 2022). Although this leaderboard does not directly measure the conversational quality of chat models, it does provide a useful signal to validate whether fine-tuning has introduced regressions on the base model’s reasoning and truthfulness capabilities. Across all benchmarks, we compare ZEPHYR-7B against a variety of open and proprietary models, each with different alignment procedures. To facilitate comparison across open model sizes, we group our comparisons in terms of 7B models (XWIN-LM (Team, 2023), MISTRALINSTRUCT (Jiang et al., 2023), MPT-CHAT (ML, 2023), and STABLELM-α), as well as larger models up to 70B parameters (LLAMA2-CHAT (Touvron et al., 2023), VICUNA˜ (Chiang et al., 2023), WizardLM (Xu et al.), and GUANACO (Dettmers et al., 2023)). For the chat benchmarks, we also compare against proprietary models, including CLAUDE 2, GPT-3.5-TURBO and GPT-4 (OpenAI, 2023). 4.3 DETAILS OF SFT TRAINING We train our SFT models for one to three epochs. We use a cosine learning rate scheduler with a peak learning rate of 2e-5 and 10% warmup steps. We train all models with a global batch size of 512 and use packing with a sequence length of 2048 tokens. 4.4 DETAILS OF DPO TRAINING Similar to SFT, we train our DPO models for one to three epochs. We use a linear learning rate scheduler with a peak learning rate of 5e-7 and 10% warmup steps. We train all models with a global batch size of 32 and use β = 0.1 from Eq. (1) to control the deviation from the reference model. The final ZEPHYR-7B model was initialized from the SFT model that was trained for one epoch and further optimized for three DPO epochs (see Figure 3 for an epoch ablation on MT-Bench). This paper is available on arxiv under CC 4.0 license. 1 //huggingface.co/collections/HuggingFaceH4/ zephyr-7b-6538c6d6d5ddd1cbb1744a66 Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix 4 EXPERIMENTAL DETAILS We conduct all of our fine-tuning experiments using Mistral 7B (Jiang et al., 2023), which is the current state-of-the-art base LM at the 7B parameter scale, and matches the performance of much larger models like LLaMa 34B on many NLP benchmarks. We use the Transformer Reinforcement Learning (TRL) library for fine-tuning (von Werra et al., 2020), in conjunction with DeepSpeed ZeRO3 (Rajbhandari et al., 2020) and FlashAttention-2 (Dao, 2023) to optimize memory and improve training speed. All models are trained with the AdamW optimizer and no weight decay. We did not experiment with parameter-efficient techniques such as LoRA (Hu et al., 2021), but expect similar results to hold with these methods. All experiments were run on 16 A100s using bfloat16 precision and typically took 2-4 hours to complete. For the full set of hyperparameters and instructions on how to train the models, see: //github.com/huggingface/alignment-handbook. 4.1 DATASETS We focus on two dialogue datasets that have been distilled from a mix of open and proprietary models, and have previously been shown to produce strong chat models like the UltraLM (Ding et al., 2023): • UltraChat (Ding et al., 2023) is a self-refinement dataset consisting of 1.47M multi-turn dialogues generated by GPT-3.5-TURBO over 30 topics and 20 different types of text material. We initially ran dSFT over the whole corpus, but found the resulting chat model had a tendency to respond with incorrect capitalization and would preface its answers with phrases such as “I don’t have personal experiences”, even for straightforward questions like “How do I clean my car?”. To handle these issues in the training data, we applied truecasing heuristics to fix the grammatical errors (approximately 5% of the dataset), as well as several filters to focus on helpfulness and remove the undesired model responses. The resulting dataset contains approximately 200k examples. • UltraChat • UltraFeedback (Cui et al., 2023) consists of 64k prompts, each of which have four LLM responses that are rated by GPT-4 according to criteria like instruction-following, honesty, and helpfulness. We construct binary preferences from UltraFeedback by selecting the highest mean score as the “chosen” response and one of the remaining three at random as “rejected”. We opted for random selection instead of selecting the lowest-scored response to encourage diversity and make the DPO objective more challenging. As noted above, this step is computed offline and does not involve any sampling from the reference model. • UltraFeedback We make the pre-processed datasets available on the Hugging Face Hub.1 4.2 EVALUATION Our main evaluations are on single-turn and multi-turn chat benchmarks that measure a model’s ability to follow instructions and respond to challenging prompts across a diverse range of domains: • MT-Bench (Zheng et al., 2023) is a multi-turn benchmark that consists of 160 questions across eight different areas of knowledge. In this benchmark, the model must answer an initial question, and then provide a second response to a predefined followup question. Each model response is then rated by GPT-4 on a scale from 1-10, with the final score given by the mean over the two turns. • MT-Bench • AlpacaEval (Li et al., 2023) is a single-turn benchmark where a model must generate a response to 805 questions on different topics, mostly focused on helpfulness. Models are also scored by GPT-4, but the final metric is the pairwise win-rate against a baseline model (text-davinci-003). • AlpacaEval We also evaluate ZEPHYR-7B on the Open LLM Leaderboard (Beeching et al., 2023), which measures the performance of LMs across four multiclass classification tasks: ARC (Clark et al., 2018), HellaSwag (Zellers et al., 2019), MMLU (Hendrycks et al., 2021), and Truthful QA(Lin et al., 2022). Although this leaderboard does not directly measure the conversational quality of chat models, it does provide a useful signal to validate whether fine-tuning has introduced regressions on the base model’s reasoning and truthfulness capabilities. Across all benchmarks, we compare ZEPHYR-7B against a variety of open and proprietary models, each with different alignment procedures. To facilitate comparison across open model sizes, we group our comparisons in terms of 7B models (XWIN-LM (Team, 2023), MISTRALINSTRUCT (Jiang et al., 2023), MPT-CHAT (ML, 2023), and STABLELM-α), as well as larger models up to 70B parameters (LLAMA2-CHAT (Touvron et al., 2023), VICUNA˜ (Chiang et al., 2023), WizardLM (Xu et al.), and GUANACO (Dettmers et al., 2023)). For the chat benchmarks, we also compare against proprietary models, including CLAUDE 2, GPT-3.5-TURBO and GPT-4 (OpenAI, 2023). 4.3 DETAILS OF SFT TRAINING We train our SFT models for one to three epochs. We use a cosine learning rate scheduler with a peak learning rate of 2e-5 and 10% warmup steps. We train all models with a global batch size of 512 and use packing with a sequence length of 2048 tokens. 4.4 DETAILS OF DPO TRAINING Similar to SFT, we train our DPO models for one to three epochs. We use a linear learning rate scheduler with a peak learning rate of 5e-7 and 10% warmup steps. We train all models with a global batch size of 32 and use β = 0.1 from Eq. (1) to control the deviation from the reference model. The final ZEPHYR-7B model was initialized from the SFT model that was trained for one epoch and further optimized for three DPO epochs (see Figure 3 for an epoch ablation on MT-Bench). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv 1 //huggingface.co/collections/HuggingFaceH4/ zephyr-7b-6538c6d6d5ddd1cbb1744a66

Zephyr: Direct Distillation of LM Alignment: Experimental Details

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix A APPENDIX A.1 QUALITATIVE EXAMPLES To qualitatively compare the responses from our dSFT and dDPO models, we choose prompts from a few domains of MT-Bench, as well as some adversarial prompts to test each model’s capability to follow instructions with false premises or harmful intent. Completions for the adversarial prompts were generated with nucleus sampling(top-p = 0.95) and T = 0.7. A.2 SFT IS A REQUIRED STEP BEFORE DPO In Table 3 we ran an ablation to see whether SFT is necessary prior to the DPO step. We observed a significant reduction in performance in both the MT-Bench and AlpacaEval scores when the SFT step is skipped. After a qualitative evaluation of the MT-Bench generations, we observe that the pure DPO model struggles to learn the chat template: This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix A APPENDIX A.1 QUALITATIVE EXAMPLES To qualitatively compare the responses from our dSFT and dDPO models, we choose prompts from a few domains of MT-Bench, as well as some adversarial prompts to test each model’s capability to follow instructions with false premises or harmful intent. Completions for the adversarial prompts were generated with nucleus sampling(top-p = 0.95) and T = 0.7. A.2 SFT IS A REQUIRED STEP BEFORE DPO In Table 3 we ran an ablation to see whether SFT is necessary prior to the DPO step. We observed a significant reduction in performance in both the MT-Bench and AlpacaEval scores when the SFT step is skipped. After a qualitative evaluation of the MT-Bench generations, we observe that the pure DPO model struggles to learn the chat template: This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Stories, News & Mentions about BENCHLING

Benchling’s Stories on HackerNoon

Benchling's News & Mentions