We spent this week mostly polishing the Dashboard and improving our SQL compiler. The highlights are that the dashboard is now more performant and intuitive, the SQL compiler is more interpretable and robust, and the whole app should be more stable. In particular, the messages section of the modal has been made a lot more expressive and clear.

Improvements

• We fixed some overlap issues in the dashboard so now the item you're selecting is always on top.
• Multiselection states are now properly synced to other users in the dashbord.
• The Channel Compiler will tell you which examples and definitions it's using when it compiles your SQL.
• The Compiler also deals better with predictions and statistical tests now.
• The SQL editor in the question modal is now more robust and saves your state properly.
• The dashboard modal shows which data source was used.
• Negative numbers are formatted properly on question answers.

New dashboard

We’ve been listening very closely to your feedback and working over the past few weeks on significant improvements. We’ve totally changed how we generate SQL, refactored the entire dashboard, and redesigned how you interact with visualizations. We'd love for you to try out the new version and share your feedback with us!

Improvements

Here's a quick overview of the exciting changes in the new version of Fluent:

• Improved SQL performance: We’ve totally reimagined how we generate SQL meaning that it’s more accurate, more contextual and far less likely to hit OpenAI’s rate limit. If we do hit the rate limit, this no longer wipes the question history.
• Interact history: Refine is now interact and you have access to the full version history of each question - revert back to any step or create a new checkpoint to explore a certain direction. You can include as many interact checkpoints on a dashboard as you like.
• New question modal design: We’ve updated the design of the question modal (the expanded card view) to emphasise the visualisation and interact panel.
• Manipulate the visualisations and data format: From the data table, you can change the X and Y axes, format data as numbers, currencies, and much more, as well as the colour of the graph directly.

Channel makes predictions, and runs statistical tests!

We’ve been improving the Channel Compiler (what we call the code that takes your question and turns it into SQL, graphs, etc.) and it can now do predictions. For instance, in this image we refined our graph of crime types in 2020 and 2021 by asking “predict, based on 2020 and 2021, what 2023 will be for each and map in a new column”. We’ve also been experimenting with asking Channel to do statistical tests, for instance in our analytics dataset we did a correlation test to check whether Channel is faster when it has to generate less SQL (it is!), and you can ask Channel to do things like answer “Is the amount of revenue we generate correlated with the day of the week”.

Have a go with asking Channel to either predict future revenue, user counts, or other interesting business variables and let us know how you get on!

Redshift and Databricks integrations

We now support AWS Redshift (either the serverless or the provisioned version) and Databricks, so if you have any data sources which are hosted on either of these platforms let us know and we can get those set up for you! Also, please let us know if there are any data sources you really want us to add support for!

Improvements

• We added support for Azure buckets as a storage backend for query results. This means that instead of query results being stored in our storage they can be stored in your storage if you use AWS, Google Cloud, or Azure.
• We fixed a bug in the data source configuration page where tables with more than a dozen or so columns were hidden.
• We fixed a bug where all user invites were broken.
• On the questions page you can now filter questions by whether or not they’ve been double checked.
• We updated some iconography to make Channel look nicer.

Double Checking!

There are now three different user roles (Admin, Data User, Business User) and anyone can request a Double Check from all Admins and Data Users. If the data generated is correct and answers the question, then they can mark the answer as Double Checked and everyone who sees that answer from then on can have the confidence that the answer is correct. Channel will only learn from Double Checked answers so you’ll have more control over the answers Channel generates.

Weekly Roundup

Once a week, you’ll now get an email highlighting the top things that happened in Channel, how many questions were asked, what the most popular dashboard was, and things like that.

Improvements

• We fixed a bug in the Snowflake Schema generation step of our pipeline.
• Improved the way we measure prompt length when using OpenAI models.
• The step of our pipeline that validates WHERE clauses has been improved and should be more robust.
• We improved the icon that we use for dashboards - not a map any more!

A week of polish, bug fixes, and performance improvements so no major features to announce. Come back next week!

Improvements

• We have this new Changelog website!
• Migrated more pipeline steps over to gpt-3.5-turbo and gpt-4.
• Improved the performance of table selection when generating question answers. This will make question answering 100x faster on larger data sources (over 100 tables).
• Fixed a bug where dashboards would crash if they included a bar chart.
• Snowflake data warehouses will now automatically resume if you ask a question.
• The loading spinner should appear less often now. There were a load of reasons why you’d get kicked back there and they should all have been addressed now.
• The dashboard title will now be synced across all users.
• There is now a delete button on the right-click menu in the dashboard.
• Loads of other improvements to prompts to increase robustness and accuracy.

Refinement

With refinement, you can now ask follow up questions to drill down into exactly the data you need. The way it works is you first ask a question as normal, and then if the data you get back isn’t quite what you’re looking for you can refine it to filter or change the results to properly answer your question.

Coming soon is refinement history, the ability to refine as new (so you can, for instance, refine a question your team mate asked) and other ways to slice your data.

Data export

You can now get the results of a query as a CSV, so you can use your data in Excel, or whatever other tools are part of your workflow. It can take a few seconds to generate the data if it’s a complicated query so when you use this feature your download will be queued and you’ll get an email when it’s ready! To use this, open any card on the dashboard and go to the downloads tab, or press the download button in the top right actions.

Improvements

• Began migrating over our SQL generation pipeline to use gpt-4 and gpt-3.5-turbo instead of gpt-3!
• Improved the design of the question card, so that important actions are always visible in the top left.
• Fixed a bug where a question would get deleted as you were typing it, when the autocomplete options popped in.
• The table view got improved, and now complicated JSON results won’t break the formatting there.
• We fixed a load of bugs around out dbt integration, so now it should be more robust. For anyone that hasn’t used this, if you connect up a Snowflake or BigQuery data source then you can also add the dbt repo used to generate it. At the moment we only support adding the entire repo as a .zip file, but we’re thinking about how to improve this in the future - probably by adding a GitHub integration.
• The UX for selecting a data source for a question got improved - now the most recently added data source is the default so hopefully no-one accidentally selects DVD Rentals again!
• Our Custom Storage Bucket feature got improved - now it’s easier to connect up your own storage bucket for query results. Let us know if you’re interested in trying this.