Blog posts by Junte's Blog

Blog posts by Junte's Blog2024-08-26T16:13:51.0000000Zhttps://world.optimizely.com/blogs/juntes-blog/ Optimizely World How to Create AI-driven Chatbots with Optimizely GraphAlan Turning in the 1950s hypothesized that a computer program could interact with humans. In 1966, the first chatbot was created at MIT called ELIZA, which <a href="https://web.njit.edu/~ronkowit/eliza.html">acts like a therapist</a>. Since then, we see that chatbots have evolved from using rules-based natural language processing to Conversational AI, including the use of generative AI (GenAI) with the ChatGPT boom.  "Hi Siri, can you find me a pizzeria?" – this is an example of Conversation AI in action. It has become a buzzword with the advent of increasingly more powerful computing power to process huge amounts of data, breakthroughs in Artificial Intelligence (AI) and the adoption of mobile devices offering speech input with Apple's Siri, Amazon's Alexa, IBM's Watson Assistant or Google Home. Conversational AI can improve our access to information by making it easier and faster to interact with online systems, either by text or speech. For example, IBM’s Watson question-answering system won the TV game-show Jeopardy! by beating humans at answering questions in 2011. Not surprisingly, when you have an online platform with a lot of curated content, trying to activate and get more of your content consumed with improved access using AI-driven chatbots becomes a tempting option. In this blog post, I will explain the basics, and present a start towards setting up a chatbot for your platform and your content with Optimizely Graph with as minimum code as possible. <h2>Conversational AI</h2> Conversation AI allows easy access to various services and information via conversation and is fundamental to natural user interfaces. These user interfaces can be <a href="https://www.uxmatters.com/mt/archives/2021/01/why-mobile-apps-need-voice-interfaces.php">text based, but also voiced based</a> with the <a href="https://www.statista.com/statistics/277125/share-of-website-traffic-coming-from-mobile-devices/">increasing use</a> of mobile devices. There have been publicly available conversational systems going back many decades. For example, the precursors of today's chatbot systems relied heavily on hand-crafted rules and are very different from the data-driven conversational AI systems of today. We see that breakthroughs in deep learning (DL) and reinforcement learning (RL) are applied to conversational AI with generative AI driven by so called Large Language Models (LLMs).  Conversational AI provides concise, direct answers to user queries based on rich knowledge drawn from various data sources including text collections such as Web documents and pre-compiled knowledge bases such as sales and marketing datasets. Search engine companies, including Google, Microsoft and Baidu, have incorporated QA capabilities into their search engines to make user experience more conversational, which is particularly appealing for mobile devices. Instead of returning ten blue links, the search engine generates a direct answer to a user query. This is in particular useful for informational queries, where the intent of the user is to look for information and get an answer to a question quickly. For websites that aim to get greater chances of conversions and stickiness, we see a chatbot present on the landing page. Example, <a href="https://www.salesforce.com/">Salesforce</a> but also <a href="https://www.optimizely.com/get-started/">Optimizely has one</a>. That does not mean that they are always driven by Conversational AI. Chatbots are also used within other channels such as WhatsApp or perhaps even your car, where users can interact with your content, while not having loaded or even seen your site. <h2>Retrieval Augmented Generation (RAG)</h2> Retrieval-Augmented Generation (RAG), <a href="https://arxiv.org/pdf/2005.11401">coined in 2005</a> decades before the GenAI boom, is an approach to generative AI that combines the strengths of traditional information retrieval systems (search engines) with the capabilities of generative large language models (LLMs).  This <a href="https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/">blog post</a> has more background information and history about RAG. By combining this extra knowledge with its own language skills, the AI can write text that is more accurate, up-to-date, and relevant to your specific needs. It reduces the problem of incorrect or misleading information ("hallucination"). It offers transparency to the model so the answers can be checked by its sources. Models have access to the most current, reliable facts, and that users have access to the model’s sources, ensuring that its claims can be checked for accuracy and ultimately trusted. RAG also reduces the need for users to continuously train the model on new data and update its parameters as circumstances evolve. RAG can lower the computational and financial costs of running LLM-powered chatbots.  It consists of 2 steps: <ol> <li>Retrieval: collect the relevant information given a query from a system with your data. This is where Optimizely Graph can be a solution. </li> <li>Generation: The pre-processed retrieved information is then seamlessly incorporated into the pre-trained LLM. This integration enhances the LLM's context, providing it with a more comprehensive understanding of the topic. This augmented context enables the LLM to generate more precise, informative, and engaging responses. </li> </ol> <h2>Putting it to Action with Optimizely Graph</h2> <h3>Overview</h3> In my previous blog post, "<a href="/link/847c0fb0d6f54a8e836b3fd0b9dd28b3.aspx">Do you know what I mean? Introducing Semantic Search in Optimizely Graph</a>", I have introduced the semantic search feature in Graph. It greatly helps us in returning more relevant results to the top by better "understanding" the query, so the answer generated by the LLM will be more accurate and complete to the user. The RAG flow with Optimizely Graph is depicted here: <img src="/link/e9b3c38b7a2f401aa1876aba605b02b8.aspx" width="863" alt="RAG with Optimizely Graph" height="703" /> Besides Optimizely Graph, we can use a cloud service that hosts LLMs with APIs. In my example, I have used <a href="https://groq.com/">Groq</a>, since it has a free trial and has public libraries in C# and Python with good documentation, so it's possible to try it first. Other very well known services are OpenAI or its Azure variant, Amazon Bedrock, Google's Vertex AI, etc. You can find the full code which is open source for my implementation in <a href="https://github.com/episerver/optimizely-graph-python-sdk">this Github repository</a>.  <h3>Setup</h3> <ul> <li>We can setup <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/docs/synchronize-content-data#authorization">Optimizely Graph by using basic authentication</a>. </li> <li>We can create the GraphQL schema types with <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/docs/synchronize-content-types">an API as described here</a>, and you will find an example for this demo here. Also see my tutorial with <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/docs/understanding-your-data">an another example with IMDb data</a> that illustrates the use of aggregating on information from <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/docs/synchronize-content-types">multiple sources in Graph</a> using <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/docs/joins-with-linking">joins</a>, and the use of <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/docs/semantic-search">semantic search</a>.</li> <li>We use the GQL Python package to interface with Optimizely Graph, and use the <a href="https://docs.developers.optimizely.com/platform-optimizely/v1.4.0-optimizely-graph/reference/post-graphqlv2handler">Content V2 endpoint</a> to send GraphQL queries to Graph. </li> <li>We use the Groq Python package to interface with the Groq APIs. In this blog post, we will use <a href="https://console.groq.com/docs/api-reference#chat-create">Groq's completion API.</a> </li> <li>We use the <a href="https://www.nltk.org/">NLTK</a> package, which is used for natural language processing tasks, to do some preprocessing of the search text sent to Graph, so we can get more accurate results ranked to the top.</li> </ul> <h3>Sync Data to Graph</h3> We can sync data to Graph by modeling the schema with content types first. The content types consist of the different types of content, the properties (field name and type), and relationship between the properties. We model based on a small <a href="https://github.com/vatsalsaglani/GraphRAG4Rec/blob/main/src/imdb/data/imdb_top_100.json">IMDb dataset created here</a>. This is how we can define the data in Optimizely Graph, which consists of the metadata of popular movies extracted from IMDb: <pre class="language-javascript"><code>{ "label": "IMDB2", "languages": [ "en" ], "contentTypes": { "Record": { "abstract": true, "contentType": [], "properties": { "ContentType": { "type": "[String]" } } }, "Movie": { "contentType": [ "Record" ], "properties": { "id": { "type": "String" }, "ContentType": { "type": "[String]" }, "genre": { "type": "[String]", "searchable": true }, "year": { "type": "String" }, "title": { "type": "String", "searchable": true }, "certificate": { "type": "String" }, "overview": { "type": "String", "searchable": true }, "director": { "type": "String", "searchable": true }, "cast": { "type": "[String]", "searchable": true } } } } }</code></pre> Then we can proceed by massaging the data to the jsonl format that allows it to be synced to Graph, and then sync it. This is how we could do it: <pre class="language-python"><code>SOURCE = "imdb2" HOST = os.getenv('HOST', "https://cg.optimizely.com") SCHEMA_SYNC_ENDPOINT = "{}/api/content/v3/types?id={}".format(HOST, SOURCE) DATA_SYNC_ENDPOINT = "{}/api/content/v2/data?id={}".format(HOST, SOURCE) AUTH_TOKEN = os.getenv('AUTH_TOKEN', "<TOKEN>") HEADERS = { 'Content-Type': 'text/plain', 'Authorization': 'Basic ' + AUTH_TOKEN } SCHEMA_FILE = "models/content_types.json" MOVIE_FILE = "data/imdb_top_100.json" # reset data def reset_data(): response = requests.request("DELETE", DATA_SYNC_ENDPOINT + "&languages=en", headers=HEADERS) # load schema def load_schemas(): with open(SCHEMA_FILE) as f: schema = json.dumps(json.load(f)) response = requests.request("PUT", SCHEMA_SYNC_ENDPOINT, headers=HEADERS, data=schema) # load the data def load_data(source, content_type, language): with open(source) as f: contents = json.load(f) bulk = "" for i, content in enumerate(contents): content["ContentType"] = ["Record", content_type] content["Status"] = "Published" content["_rbac"] = "r:Everyone:Read" content["__typename"] = content_type content["genre___searchable"] = content.pop("genre") content["title___searchable"] = content.pop("title") content["overview___searchable"] = content.pop("overview") content["director___searchable"] = content.pop("director") content["cast___searchable"] = content.pop("cast") content.pop('llm_text', None) bulk += "{\"index\": { \"_id\": \"" + source + str(i) + "\", \"language_routing\": \"" + language + "\" }}\n" + simplejson.dumps(content, ignore_nan=True) if i != len(contents)-1: bulk += "\n" response = requests.request("POST", DATA_SYNC_ENDPOINT, headers=HEADERS, data=bulk) reset_data() load_schemas() load_data(MOVIE_FILE, "Movie", "en")</code></pre> <h3>Rewrite the Question by Augmentation</h3> We apply co-reference resolution to questions where there are ambiguous pronouns in a question that are used in a follow-up question. We rewrite the question by augmentation, so a more concrete question can be asked to Optimizely Graph and the LLM can give us the right answer. We can use the LLM to do this (as <a href="https://thenewstack.io/improving-chatgpts-ability-to-understand-ambiguous-prompts/">this blog post also explains</a>), so we are using the LLM to rewrite the questions, and then use it again to get answers. This is a concept called prompt engineering, where we instruct the LLM what to do, and feed it with examples, so it learns.  Image this conversation: <ul> <li>Q: Who is James Cameron?</li> <li>A: James Cameron is a director of movies.</li> <li>Q: What movies did he direct?</li> <li>A: James Cameron directed the following movies: 1. Titanic (1997) 2. Avatar (2009) 3. Terminator 2: Judgment Day (1991) 4. Aliens (1986) 5. The Terminator (1984)</li> </ul> Without co-reference resolution, "he" cannot be resolved and Optimizely Graph cannot return the right information. So "What movies did he direct?" will be augmented as "What movies did James Cameron direct?". This is how it could be implemented: <pre class="language-python"><code>def rewrite_question(question, previous): post_prompt = "Don't explain your answers. Don't give information not mentioned in the CONTEXT INFORMATION." chats = [] if previous: chats.insert(0, {"role": "system", "content": "You only rewrite the question. Do not change the question type. Do not explain. You do not know anything about movies or actors. Return one Question with only coreference resolution based on the Context. Example, replace \"he\" with a name. If you cannot apply coreference resolution, then just return the original Question without any changes."}) chats.append({"role": "user", "content": f"Context: {previous}\n\nQuestion: {question}" + post_prompt}) rewritten_question = get_chat_completion(chats=chats) return rewritten_question return question </code></pre> <h3>Query Optimizely Graph with Semantic Search</h3> Now we have synced the data to Optimizely Graph. Time to query. We can define a GraphQL query template, where we match and rank on the searchable fields using semantic search. Additionally, we apply some weights to some fields. GraphQL returns to us a dictionary of movies with properties, which it can use to return a meaningful answer. This is how it could look like: <pre class="language-javascript"><code>{ Movie( locale: en where: { _or: [ { cast: { match: "sigourney weaver?", boost: 2 } } { director: { match: "sigourney weaver?", boost: 3 } } { title: { match: "sigourney weaver?", boost: 10 } } { _fulltext: { match: "sigourney weaver?", fuzzy: true } } ] } orderBy: { _ranking: SEMANTIC } limit: 10 ) { items { cast director overview title genre year } } }</code></pre> Stopwords (common words like "is", "the", "a", etc) can introduce noise in the semantic search query and return undesired results to the top. To improve the queries because we have a very small dataset with little content and not a normal distribution of words, we do some simple pre-processing by using the Natural Language Toolkit package to remove stopwords. For larger datasets, this step may not be needed. The retrieval stage is very important in RAG as it determines the quality of answers and the likelihood of returning "I don't know". We will continuously improve the matching and ranking in Optimizely Graph, and allow you to introduce more customization as well, as I originally posited in my <a href="/link/24e13898fe1b4ca0a1aa0e0a33693b81.aspx">first blog post</a> "Why Optimizely Graph is Search as a Service". <h3>Use a LLM to Generate an Answer From Graph</h3> Now we got results from Graph. We can feed this to Groq. We store the chat history and reuse it as context for the LLM. Saving the chat history and reusing it as the prompt context improves the answers, since more context is used. The idea is that a chat session is preserved. However, the context has a size limit. In my example, I have set the limit of the chat history to 5. Note that my example works for a single user in a single session. In case of multiple users and multiple sessions, the chat history needs to be stored and retrieved per user as well.  We use Meta's <a href="https://ai.meta.com/blog/meta-llama-3/">llama3-8b-8192</a> model and the chat completions API. To make sure we only get answers based on the context, we give instructions to the LLM. This is what we instruct our model to do: <div> <div> <ul> <li>You are a movie expert. You return answers as full sentences. If there is an enumeration, return a list with numbers. If the question can't be answered based on the context, say "I don't know". Do not return an empty answer. Do not start answer with: "based on the context". Do not refer to the "given context". Do not refer to "the dictionary". You learn and try to answer from contexts related to previous question.</li> </ul> </div> <div>We wrap this in a Flask app, so we can try it out in the browser with a simple UI. This is an example conversation with our RAG chatbot. It returns answers based on only the results from Optimizely Graph, but it's possible to include external information by increasing the "temperature" parameter. The format and style of answers can be configured in the prompts. Want to do more and improve? Feel free to try out and test manually, or evaluate with for example <a href="https://docs.ragas.io/en/stable/">RAGAS</a>. Only this way, you can find the sweet spot for best results.</div> <h3>Some Results</h3> We have built a simple Flask app to try out the chatbot. Here you see an example where we ask direct and unambiguous questions. <img src="/link/981d7e5d6b124ad9b937fd8c5711985e.aspx" width="620" alt="RAG with Optimizely Graph" height="943" style="border-style: none;" /> And what about using follow-up questions and putting co-reference resolution into practise? A question like "can you give me a summary of it?" will be rewritten as "can you give me a summary of The Avatar" Here you see an example longer conversation where we see the importance of augmenting questions.  <img src="/link/4525b621c35e41c4a21f002ccb47f9fc.aspx" width="620" alt="Optimizely Graph RAG co-reference resolution 1" height="655" style="border-style: none;" /><img src="/link/4503d1d7bfa9442c99a48186d12694a4.aspx" width="620" alt="Optimizely Graph RAG co-reference resolution 2" height="629" style="border-style: none;" /> <h2>Wrap Up</h2> </div> I have explained some history and the background of LLM-powered chatbots using an approach called Retrieval Augmented Generation (RAG). With an example implementation, I have show-cased how we can use Optimizely Graph to implement RAG on your content. Adding a chatbot next to the search box on your site  — all driven by Optimizely Graph — can be possible. You can use this as a reference, treat it as a starter and extend/improve on it to add an actual production-ready chatbot on your site that is using Optimizely Graph. The code for this example implementation is open source, and you can <a href="https://github.com/episerver/optimizely-graph-python-sdk/tree/main">find it here</a>.  Optimizely Graph allows you to aggregate on your CMS content and other data with GraphQL, including using advanced search functionality driven by AI, so you can deliver it to any channel, app or device. It is part of <a href="https://www.optimizely.com/company/press/optimizely-one/">Optimizely One</a>. It is included in the Optimizely PaaS offering as well as the CMS SaaS.2024-08-26T16:13:51.0000000Z

Blog post

Do you know what I mean? Introducing Semantic Search in Optimizely GraphIt seems not long time ago that search engines allowed users to search beyond strings and match on words. Understanding the basic structure of a natural language was good enough for search engines to move beyond string matching (like done in relational databases) and match on so-called tokens. Often, these tokens are words. A word is a basic element of a language that has meaning. However, search engines were not intelligent enough to understand the meaning of these words. Instead of matching on a very long string, it would be matching on tokens. Basically, all tokens were treated as a bag of words, where we did not assume any relation between these words. A naïve, but very effective approach in information retrieval (IR, or the science behind search). But with the advent of deep learning and other AI advancements, we are seeing new developments in search engines as well in the area of semantic search. In this post, I will try to explain the history behind the technology, the origin of the most important terminology used in this domain, and what this technology is. And how we support this in Optimizely Graph, or Graph in short. <h2>How a full-text search engine works in a nutshell</h2> Full-text search engines store text and return results efficiently. Key features are advanced matching on these texts, blazing retrieval speed, and intelligent scoring of results, so that users get most relevant information as fast as possible.     Data is stored by creating tokens. Tokens created for indexing are commonly stored in full-text search engines with a special data structure called the inverted index. Basically, it is a special table optimized for quick lookups. The index is called inverted, because the tokens (and not the documents) are stored in a column store with extra dimensions, like the reference to the documents containing the token, its frequency in a document, the positions of a token in a stream of tokens, etc. Here we see a low-level visualization of this in<a href="https://lucene.apache.org/"> Lucene</a>, which is an open-source core search library that drives a lot of the site search on the web. We see that content and its metadata is indexed into special files, and together these are called a index segment. An index consists of these segments.   <img src="/link/7987f71439044ff0a1d652e6d4900e7a.aspx" width="688" alt="Datastructure used in Lucene" height="215" /> Crucially here, is that these dimensions are used to represent a token in a vector space. We speak then of term-based vectors, or sparse embeddings, because there can be one (token present or not present) or many dimensions, and since most documents contain only a small subset of the full vocabulary, these vectors are considered sparse since zero values will be plentiful. Note that uninverted stores (so called doc values) are also used as <a href="https://blog.parse.ly/lucene/">explained in this blog post</a>, for example to efficiently create facets and drive analytics. Anyway, the sparse vector embeddings are used to match and most importantly, rank the results. In 1975, <a href="https://dl.acm.org/doi/10.1145/361219.361220">G. Salton et al.</a> came up with the vector space model for indexing. This was purely an algebraic approach, but this was extended with a probabilistic relevance model in the 1970s and 1980s by Stephen E. Robertson, Karen Spärck Jones, and others and eventually <a href="https://trec.nist.gov/pubs/trec3/papers/city.ps.gz">presented it in 1994 at TREC</a> (a conference where information retrieval researchers work on text retrieval methodologies) as Okapi BM25, or BM25 in short. The basic idea of BM25 is that the search engine considers the frequency of a word, the number of documents containing a word (with the idea to make more frequent terms ranked lower), document length normalization, and query term saturation to mitigate the impact of very frequent words. It is used till now as the baseline of relevance ranking for full-text search and the default ranking used in solutions like Elasticsearch and Apache Solr. And not surprisingly, Optimizely Graph uses BM25 also as the default relevance ranking model. <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.1.0-search-and-navigation/docs">Search & Navigation</a> though, is still using the vector space model from G. Salton et al. That is not to say that BM25 is the best model out there. A plethora of research papers have shown that using a pure probabilistic model with <a href="https://www.elastic.co/blog/language-models-in-elasticsearch">language models</a> can outperform BM25. And ranking results is much more sophisticated than just statistics and probability theory on the tokens that you have stored. Search has been significantly improved by incorporating many other signals in ranking models. Can anyone still remember <a href="https://en.wikipedia.org/wiki/PageRank">PageRank</a>? Google made web search commodity because of the highly successful application of this, where the number of incoming links to a page could be used to boost, so frequently linked pages showed up higher. Google has said that more than 200 signals are used for their search. Indeed, using signals like these are part of artificial intelligence, which helps to improve getting the most relevant content shown on the screen of the user. <h2>AI comes knocking at the door</h2> We thought that web search was a solved problem for most users. Users seem satisfied with what they got from Google. However, with the boom happening with generative AI, made possible with the huge improvements in deep learning, new ways to interact with information and new insights that we can gain have been unlocked. Instead of scrolling through a list of text, chat bots can directly deliver the answer that you were looking for. Talking to a device and get a satisfying verbal response from that device is no longer science-fiction. Key to generative AI technology is the concept of transforming text into numbers. We need this, because machine learning models understand numbers, but not words. We can train on a lot of text and build so called Large Language Models (LLMs), which consists of numerical representations. To understand this core technology, I can recommend the <a href="https://ig.ft.com/generative-ai/">visual explanation on transformers here</a>.  A word can be associated with multiple dimensions based on linguistic features, which are represented as vectors, and combined they are called an embedding. They are coordinates in the vector space. The number of dimensions will be limited because only useful ones are selected and used. That is why we call these numbers also dense vector embeddings. This is different compared to the sparse vector embeddings we calculate from the tokens stored in an inverted index.  This example shows how we can visualize embeddings computed from Star Wars in the vector space (courtesy <a href="https://medium.com/@marcusa314/visualizing-words-377624cb20c7">@Marcus Alder</a>). So, for example, based on the same dimensions, we will see that words that have similar dimensions will be located closer to each other in the vector space. A word like “cola” will be in a shorter distance to “cold beverage” than “hot chocolate milk”. Or in this visualization, "Anakin" will be closer to "Luke" than "Endor". <img src="https://miro.medium.com/v2/resize:fit:720/1*Y7K_-6HZqbii1V8Ms6SALQ.gif" /> <h2>Why we need semantic search</h2> The idea of semantic search is not new, but it has become a new paradigm in industry. It is also called (though not necessarily synonyms) neural search, vector search, dense neural retrieval, etc. The idea is that the intent of searchers is understood by the search engine by understanding the meaning of the words that it gets as input. It is made possible now with LLMs, which give us the embeddings. And not to forget, crucially, we have vector search support in search engines to do vector similarity search using similarities like Euclidian, dot-product or Cosine. Vector search is here K-nearest neighbor search on the embeddings. With approximate nearest neighbor search using <a href="https://arxiv.org/abs/1603.09320">Hierarchical Navigable Small World</a> (HNSW) graphs, we can do this at scale and efficiently with good performance. The limitation of standard full-text search with keyword matching is well known as the vocabulary mismatch problem. Relevant content may not be found by site visitors when they use the all-important search box (=3x more likely to convert), because the words that they use in their queries may not occur in the content that you have created. In the worst case, the dreadful “no results found” is shown, resulting in a missed chance to convert and perhaps even churn. Synonyms can be used to solve this problem. However, creating synonym lists is time-consuming and require considerable labor though. And likely, you will always play catch-up. Moreover, with semantic search you can formulate your queries in natural language, enabling better ways to deal with queries. Solving the problem of the vocabulary mismatch is one of the quick wins. Improving the relevance ranking is another. We can boost relevant content to the top by combining scores from semantic search with standard BM25 relevance ranking. It is crucial that we do not only return results, but also return them in the best relevant order. The result that you see first has a greater chance to be clicked on. Experiments (see for example <a href="https://blog.vespa.ai/improving-zero-shot-ranking-with-vespa-part-two/">here</a>, <a href="https://opensearch.org/blog/semantic-science-benchmarks/">here</a> and <a href="https://towardsdatascience.com/text-search-vs-vector-search-better-together-3bd48eb6132a">here</a>) have shown that combining both keyword-based (lexical) ranking with BM25 and vector-based search, an approach called hybrid search, works better than doing only keyword search or vector-search. And using pure vector search is not always beating BM25, especially in special verticals. Another common application of semantic search is reducing very convincing looking, but false information returned by chatbots driven by LLMs. A phenomenon known as hallucination. A very popular approach now is using <a href="https://research.ibm.com/blog/retrieval-augmented-generation-RAG">Retrieval Augmented Generation</a> (RAG). The quality of LLMs is improved by using RAG. It offers transparency to the LLMs so the answers can be checked by its sources. Models have access to the most current, reliable facts, and users have access to the model’s sources, ensuring that its claims can be checked for accuracy and ultimately trusted. Imagine adding a chatbot powered by your content on your site using Optimizely Graph in a not too distant future. Wouldn't that be awesome? <h2>Semantic search in Optimizely Graph</h2> Our implementation of semantic search with Graph has been released. We offer it as-a-service and have done the heavy-lifting for you. The feature is experimental now, which means that the results you get now may change in the future, likely for the better as we will try to improve on it. You can find the documentation on <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/semantic-search">Optimizely Graph's semantic search</a> here. We use a pre-trained model that has been trained on a plethora of well-known websites such as Wikipedia and Reddit, using over 1 billion sentences. The model has been trained on English, and works on English content.  You can start by synchronizing your content, but please define fields that you think will be useful for full-text search by setting the property as “searchable”. We support semantic search for full-text search with the field <code>_fulltext</code> and other searchable fields using the <code>contains</code> and <code>match</code> <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/graphql-schema">operators in our query language</a>. You can enable semantic search by adding the <code>SEMANTIC</code> ranking enum in the orderBy argument. That’s it! <pre class="language-markup"><code>{ Content(orderBy: { _ranking: SEMANTIC }, where: { _fulltext: { contains: "action movie" } }) { total items { Name _fulltext } } }</code></pre> And it should return results that are about an action movie, even when action movie is not mentioned in your content. For example: <pre class="language-markup"><code>{ "Content": { "total": 4, "items": [ { "Name": "Standard Page 12", "MainBody": "Wild Wild West is a 1999 American steampunk Western film co-produced and directed by Barry Sonnenfeld and written by S. S. Wilson and Brent Maddock alongside Jeffrey Price and Peter S. Seaman, from a story penned by brothers Jim and John Thomas. Loosely adapted from The Wild Wild West, a 1960s television series created by Michael Garrison, it is the only production since the television film More Wild Wild West (1980) to feature the characters from the original series. The film stars Will Smith (who previously collaborated with Sonnenfeld on Men in Black two years earlier in 1997) and Kevin Kline as two U.S. Secret Service agents who work together to protect U.S. President Ulysses S. Grant (Kline, in a dual role) and the United States from all manner of dangerous threats during the American Old West.", "_fulltext": [ "Wild Wild West is a 1999 American steampunk Western film co-produced and directed by Barry Sonnenfeld and written by S. S. Wilson and Brent Maddock alongside Jeffrey Price and Peter S. Seaman, from a story penned by brothers Jim and John Thomas. Loosely adapted from The Wild Wild West, a 1960s television series created by Michael Garrison, it is the only production since the television film More Wild Wild West (1980) to feature the characters from the original series. The film stars Will Smith (who previously collaborated with Sonnenfeld on Men in Black two years earlier in 1997) and Kevin Kline as two U.S. Secret Service agents who work together to protect U.S. President Ulysses S. Grant (Kline, in a dual role) and the United States from all manner of dangerous threats during the American Old West.", "Standard Page 12" ] }, { "Name": "Temporary Page Title", "MainBody": "The American frontier, also known as the Old West, popularly known as the Wild West, encompasses the geography, history, folklore, and culture associated with the forward wave of American expansion in mainland North America that began with European colonial settlements in the early 17th century and ended with the admission of the last few contiguous western territories as states in 1912. This era of massive migration and settlement was particularly encouraged by President Thomas Jefferson following the Louisiana Purchase, giving rise to the expansionist attitude known as \"Manifest Destiny\" and the historians' \"Frontier Thesis\". The legends, historical events and folklore of the American frontier have embedded themselves into United States culture so much so that the Old West, and the Western genre of media specifically, has become one of the defining periods of American national identity.", "_fulltext": [ "The American frontier, also known as the Old West, popularly known as the Wild West, encompasses the geography, history, folklore, and culture associated with the forward wave of American expansion in mainland North America that began with European colonial settlements in the early 17th century and ended with the admission of the last few contiguous western territories as states in 1912. This era of massive migration and settlement was particularly encouraged by President Thomas Jefferson following the Louisiana Purchase, giving rise to the expansionist attitude known as \"Manifest Destiny\" and the historians' \"Frontier Thesis\". The legends, historical events and folklore of the American frontier have embedded themselves into United States culture so much so that the Old West, and the Western genre of media specifically, has become one of the defining periods of American national identity.", "Temporary Page Title" ] }, { "Name": "Wilder Westen", "MainBody": "Wilder Westen ist eine – geographisch und historisch grob eingegrenzte – umgangssprachliche Bezeichnung für die ungefähr westlich des Mississippi gelegenen Gebiete der heutigen Vereinigten Staaten. In der auch als &bdquo;Pionierzeit“ bezeichneten Ära des 19. Jahrhunderts waren sie noch nicht als Bundesstaaten in die Union der Vereinigten Staaten aufgenommen. Im Verlauf der voranschreitenden Landnahme und Urbanisierung nahm die Besiedlung dieser Regionen vor allem durch Angloamerikaner – bzw. aus Europa stammende Immigranten – kontinuierlich zu, bis die Gebiete um 1890 in den organisierten Territorien der Vereinigten Staaten aufgingen. Symbolisch stehen die Öffnung der letzten Indianerterritorien im späteren US-Bundesstaat Oklahoma für die Besiedlung durch Kolonisten 1889–1895 durch eine Serie von Land Runs und das Massaker der United States Army an etwa 200 bis 300 Lakota am Wounded Knee Creek/South Dakota im Dezember 1890 für das Ende der Zeit des Wilden Westens. Mit diesen Ereignissen galten die Indianerkriege ebenso als abgeschlossen wie die Kolonisation der bis dahin von den Vereinigten Staaten beanspruchten Hoheitsgebiete (engl. territories) durch die aus Europa eingewanderten Siedler.", "_fulltext": [ "Wilder Westen ist eine – geographisch und historisch grob eingegrenzte – umgangssprachliche Bezeichnung für die ungefähr westlich des Mississippi gelegenen Gebiete der heutigen Vereinigten Staaten. In der auch als &bdquo;Pionierzeit“ bezeichneten Ära des 19. Jahrhunderts waren sie noch nicht als Bundesstaaten in die Union der Vereinigten Staaten aufgenommen. Im Verlauf der voranschreitenden Landnahme und Urbanisierung nahm die Besiedlung dieser Regionen vor allem durch Angloamerikaner – bzw. aus Europa stammende Immigranten – kontinuierlich zu, bis die Gebiete um 1890 in den organisierten Territorien der Vereinigten Staaten aufgingen. Symbolisch stehen die Öffnung der letzten Indianerterritorien im späteren US-Bundesstaat Oklahoma für die Besiedlung durch Kolonisten 1889–1895 durch eine Serie von Land Runs und das Massaker der United States Army an etwa 200 bis 300 Lakota am Wounded Knee Creek/South Dakota im Dezember 1890 für das Ende der Zeit des Wilden Westens. Mit diesen Ereignissen galten die Indianerkriege ebenso als abgeschlossen wie die Kolonisation der bis dahin von den Vereinigten Staaten beanspruchten Hoheitsgebiete (engl. territories) durch die aus Europa eingewanderten Siedler.", "Wilder Westen" ] }, { "Name": "Arnold Schwarzenegger", "MainBody": null, "_fulltext": [ "Arnold Schwarzenegger" ] } ] } }</code></pre> Curious about how well it works on your content? Give Optimizely Graph a try. <h2>Wrap up</h2> I have presented the technical background of semantic search, explained some key concepts and terminology. The different applications of semantic search. And how you can use it in Graph. With the introduction of semantic search, we have taken a big step in the mission to support you with AI search as previously posited in my blog post <a href="/link/24e13898fe1b4ca0a1aa0e0a33693b81.aspx">"Why Optimizely Graph is Search as a Service"</a>. So what are the next steps with AI-driven search? There are a few things. Currently, the model that we use supports English. We will soon release support for other languages, including the frequently requested Nordic languages. Another thing is improving the ranking together with standard relevance scoring. We are also looking into supporting custom models too, DIY solutions using for example LangChain, and use Graph to ground the truth in chatbots. What are the next steps besides improving on AI? Graph gets powerful as it becomes the gateway to your content. Having good search will prove to be crucial here to achieve continuous success. Creating great search experiences is an art. The importance of search analytics and <a href="https://feedback.optimizely.com/ideas/CG-I-65">tracking</a> with both online and offline experimentation will become clear as new features that show how science can meet art, how we can measure performance and experiment. This also allows us to support personalization with Graph. In a future blog post, I will bring you an update on that. Have other ideas? Feel <a href="https://feedback.optimizely.com/?project=OG">free to share or upvote on existing ideas</a>.   Optimizely Graph allows you to aggregate on your content and other data with GraphQL, including using advanced search functionality driven by AI, so you can deliver it to any channel, app or device. It is part of the Optimizely PaaS offering as well as <a href="https://www.optimizely.com/company/press/optimizely-one/">Optimizely One</a>.  2023-10-13T11:18:04.0000000Z

Blog post

Why Optimizely Graph is Search as a ServiceJohn Håkansson has already announced the public beta of a <a href="/link/386fb3fd598c4978aaf6906a1bbc1505.aspx">new Optimizely service called Content Graph</a> (now: Optimizely Graph). Jonas Bergqvist has created a <a href="/link/2aefcdfa9169413db278877740df254a.aspx">tutorial on how to get started with React and TypeScript</a>. I blogged previously about <a href="/link/b280f38c7e9d4a5d83c1eeb6f34326cb.aspx">our developer journey so far</a> in creating this service. In this blog post I want to follow-up and explain why Optimizely Graph is not only for content delivery, but that it can be the search engine for your site. <h2>Introduction</h2> As recently as the 1990s, studies showed that most people preferred getting information from other people rather than from search engines. Back then, most people also used human travel agents to book their travel or asked a librarian to find a book. However, times have changed. During the last decades, optimization of information retrieval effectiveness has driven web search engines to new quality levels where most people are satisfied most of the time, and web search has become the preferred source of information finding. Any website that continuously publishes new content, needs to have a search engine. Optimizely Graph does exactly that. It is an Optimizely SaaS solution for creating a website with search functionality using a GraphQL API that is hosted on the CDN. A very nice content delivery API using GraphQL that is platform independent. However, I would argue that Optimizely Graph primarily allows you to build an advanced search engine, and not merely building a website, as it can do much more. <h2>Why Building a Search Engine is Hard?</h2> “Consider a future device …  in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.” – Vannevar Bush, 1945 The vision of having a single system which you can access as an archive with speed and flexibility has been realized with the advent of search engines. A search engine is a complex system. Building and maintaining one is very hard and costly. From optimally storing the data in a way that you can perform sophisticated algorithms on for matching and ranking, to retrieving it as fast as possible, to processing queries that capture simple or very complicated information needs. We have off-the-shelf search engines that allow you to build your own search engine with your data. Well-known ones are Apache Solr, Elasticsearch and OpenSearch which are driven by the information retrieval library Lucene. Other lesser-known examples, but not less powerful, are for example Sphinx or Vespa. What they all have in common is that you need to spend considerable time and effort in creating an application with them like infrastructure configuration and hosting, defining/tweaking index schemas, preparing data for ingestion, and writing optimal queries. In short, a lot of preparation and testing is needed before you can create and deploy your search engine. Elasticsearch is offered as-a-service in a single-tenant environment by Elastic, and OpenSearch as well by AWS and others. Basically, this means that it is offered as platform-as-a-service. You no longer need to manage the hosting and operations yourself, but you are still responsible for the higher software layer and use this platform it in the best possible way in your application. Getting it implemented is one thing, getting it right so you get good performance and conversion rates is difficult and another cup of tea. Something that requires continuous experimentation and improvements. How can we offer a search engine from platform-as-a-service to truly software-as-a-service, where we do the heavy lifting for you? And how can we make sure you will get it right for your business context?  <h2>A Search Engine as a Service?</h2> Imagine that we can move up in the value chain, where we do all the ingestion of your content, content enrichment, configurations, tweaking and tuning of the most efficient and effective queries, and most useful ranking models for you. All that you must do to get started is adding a few lines of configuration and hitting the button. That would be truly software as a service, and in the case of Optimizely Graph, search (engine) as a service. This is exactly what we have created.  Optimizely Graph is a multi-tenant cloud service that offers you search as a service with GraphQL. We believe that GraphQL allows you to create intuitive and simple queries without a steep learning curve, because it is grounded in a strongly typed schema that offers introspection support. The query language that we developed with GraphQL allows you to query for your content created in Content Cloud, so you could create a whole site with a single GraphQL query. And it allows you to create site search by using the same API with predefined query templates to filter, match, retrieve and rank the results given keywords entered by site visitors. All you need is one endpoint and authentication keys provisioned by the DXP portal. Sounds easy right? Besides search features, we have worked hard on our infrastructure code and configurations. You do not need to worry about doing operations. We will manage that for you. That does not mean we will be constantly busy manually managing the clusters. The beating (search engine) heart of Optimizely Graph is now OpenSearch. We have spent quite some time in automation of our multi-tenant distributed platform, so we can do rolling upgrades of newer (major) versions of OpenSearch containing bug fixes, performance improvements and new features, and when necessary, upgrade all indices after the upgrade without interrupting or degrading the service. We also have auto-scaling of our distributed search engine clusters, so we will be ready for Black Friday type of scenarios. At the same time, we realize that in a multi-tenant platform, performance isolation could be needed. Have a very noisy neighbor on the same cluster? Or is there a need to have single-tenant support? We are ready. <h2>What Optimizely Graph can do as a Search Engine</h2> <h3>Ranking & Matching</h3> Optimizely Graph offers a query language that allows you to precisely filter, select and navigate the information you need. What you request is exactly what you get, preferably at the top. We have value-based ordering by fields and have state-of-the-art BM25 relevance ranking. The information will be optimally returned at blazing speed. You can rank your results very differently, but also accurately and efficiently. The results are ranked based on filtering in values in fields with wildcard support (where we optimized suffix searches) or full-text search with language analysis on fields that you want to support. We support text analysis for all languages that Search & Navigation supports, and have improved the full-text search in German and the CJK languages. This allows you to increase visitor engagement on your site by adding the all-important search box. Note that site visitors are 3x more likely to convert when they use the search box. <h3>Query with relations</h3> One of the things that we have created is a way to easily query with relations in your content. Not only within a single index with different content types, but also among indices with potentially different data sources. There is a reason why this service is called Optimizely Graph after all. One of the first relations you can query on are <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/parentchild-queries">parent-child relations</a>. This means you can use one query to get information of both a parent document and its linked child documents. The powerful thing here is that you can do filtering and full-text search and add facets as you query with this relation type -- all with a single query. Content can be queried as a graph.  <h3>Navigation</h3> Besides search, we offer different ways to do navigation. Obviously you can navigate through your search space with <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/skip-limit">pagination</a> and <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/cursor">cursors</a>, but more interesting, you can use facets. So, you can present very different views of your content by just tweaking the ranking and/or <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/facets">adding facets</a> and filters. With facets, we also offer the support of multi-select which is conventional in the e-commerce domain. This is a great navigation technique that allows you to simultaneously zoom in and zoom out on your data. Great way to distribute diverse (personalized) content with visitors across different channels, but also between systems such as other search engines using web crawlers. <h3>Search customization</h3> The challenge for us is to create a solution that is generic but also flexible enough to realize different use-cases that could be possible. “As such, there is no one-size-fits-all approach that anyone can offer you. The hot water that softens a carrot will harden an egg.” --- Clayton M. Christensen  We believe that the current set of search features allows you to build a very good search engine. We will offer more out-of-the-box search features that will be state-of-art and will make the search engine “more intelligent”, and will you to allow to opt-in to use AI search, so you will have a head start. Advances in conversational AI, such as with ChatGPT, shows that the domain of search is continuously improving and changing. I have written <a href="https://github.com/juntezhang/exploring-question-answering">a short blog post before </a>on this topic.  But we also realize that we need to give you the control required to configure the best search for your context and business cases, for example by <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/synonyms">configuring synonyms</a> and <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/boosting">field boosting</a>. We support this now. We intend to create more search features that will give you some control over the nuts and bolts, so you can customize. I will explain more in a future blog post. We will continue offering new search capabilities that will make a big difference for our customers, and improve and go beyond what we offer now. Have ideas or a wishlist? You are <a href="https://feedback.optimizely.com/?project=PGQL">very welcome to share</a> them with us. Want to see some GraphQL queries before getting started? The developer documentation of Optimizely Graph with the description of its query language with plenty of example queries <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs">can be found on our ReadMe</a>. <h2>Beta Availability</h2> Optimizely Graph is available for all Optimizely DXP customers. Feel free to <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/getting-started">get started</a>. Help us gathering bugs! Users of the beta release are encouraged to report any bugs at <a href="https://www.optimizely.com/support/?_gl=1*14bemx2*_ga*NDA0ODc3OTY2LjE2NTEyMjQzMjg.*_ga_C7SLJ6HMJ5*MTY2MjU2NTE2MC4xMi4xLjE2NjI1NjczOTAuNTIuMC4w">our support team</a>. Feature or change requests are warmly welcomed as well. You can create ideas and feedback <a href="https://feedback.optimizely.com/?project=PGQL">here</a>. 2022-12-29T14:56:17.0000000Z

Blog post

Content Graph, the Story So Far From a Developer's Perspective<h2>Introduction</h2> As content is created and managed, the volume of data grows. The need to fish for information in the ocean of data becomes crucial. “A world where everyone creates content gets confusing pretty quickly without a good search engine.” --- Ethan Zuckerman Here search engines -- and our new service Content Graph -- come into play! In this blog post, I will explain how we started our journey of creating this brand-new service in-house at Optimizely from a developer's point of view. We already have a very powerful fluent C# client API that allows you to search in your content called Search & Navigation. This is tightly coupled to the CMS and site building. As we offer a DXP with a headless CMS with the Content Cloud, there is an opportunity to de-couple and offer a new service with capabilities similar to Search & Navigation, but one that is not coupled to a programming language or framework and is easy to use by developers and even non-developers. John Håkansson has already <a href="/link/386fb3fd598c4978aaf6906a1bbc1505.aspx">announced the public beta</a> of this service called Content Graph. Jonas Bergqvist has created a <a href="/link/2aefcdfa9169413db278877740df254a.aspx">tutorial on how to get started</a> with React and TypeScript. In this blog post, I am reflecting on what we have done so far, and try to illlustrate the vision and hopes of this new service from an engineering perspective. <h2>Opportunity to Optimize</h2> We identified an opportunity to help our customers. But for us as engineers, we see that with a new service comes new technical opportunities to do things not only differently, but also in a better way. There are lessons to be learned from Search & Navigation in terms of operations but also common and not so common use-cases. We wanted to learn from our vast experience and improve as much as we can with this new service. And have a solid foundation to build on to increasingly deliver more value with quality in the future. <h3>New architecture, newer technologies</h3> As Content Graph is a new service, it has also been built up entirely from scratch at Optimizely. The GraphQL runtime is hosted on a CDN, which allows for the lowest and stable latencies possible across regions. This service interfaces with a set of interconnected containerized micro-services that are running on Kubernetes. We are also using OpenSearch, the open-source fork of Elasticsearch as a distributed search engine to store, index and retrieve data. This is also deployed as containers on Kubernetes. For all our micro-services and OpenSearch clusters, we have configured horizonal auto-scaling, and with the case of OpenSearch, we are using a Kubernetes operator. And this is all provisioned with automated pipelines with unit, end-to-end and performance tests to ensure continued quality.  This setup allows our geographical distributed teams to simplify the development, release and deployment processes --- something we are contantly improving. Also, to optimize IT costs by being smarter with resource allocation of different micro-services using horizontal auto-scaling. It makes our service more stable with high availability with self-healing in case of crash of an instance of a service. And it makes us independent of any cloud provider just in case. <h3>Operational readiness</h3> As with any new service built from scratch, one that is offered as-a-service, we need to be prepared and ready to understand what is going on, make sure everything is up and running, and offer support in case of issues. In the beta phase of Content Graph, the development team also offers operational support. We have spent considerable time on operational readiness, i.e., to improve our logging and traceability, so we can improve our monitoring and alerting capabilities in case of system issues but also to get a greater understanding of usage metrics. Dashboards with metrics have been created and alerts via email and chat have been set.  Good developer documentation is pivotal as well. We made writing developer documentation part of our development work. A user story will only be accepted as done when the documentation is there or has been updated. The <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs">developers' documentation of Content Graph</a> is kept up to date as we extend our product. This all gives us a feeling that we can be the captain on our ship, as more passengers board to join this exciting journey. <h3>Performance and experimentation</h3> Having the best possible performance is also one of our key objectives. With performance here we mean the efficiency of processing requests and returning responses with the lowest latency possible. “There are no speed limits on the road to success.” --- David W. Johnson It is a given that we will always have a hop between the CDN and our Kubernetes cluster via the internet. We reduce this by using caching in the CDN. The distribution of the queries sent to our system will likely have a long tail, but it is expected that the head of the distribution will consist of the bulk of the requests. This ensures the fastest possible performance. Content can also be frequently updated, so we will invalidate caches when there are updates. Caching will be made smarter in the future. Besides caching in the CDN, we spent time on improving the performance within the cluster by tracing the response times in the different services. Crucial here is benchmarking with performance tests. Given the same experimental conditions, we can hypothesize and tweak things --- like different configurations or different implementations --- that we think will be an improvement and compare the results. And gains of every hundred milliseconds count! Besides software development, we also experiment in an Agile way --- incrementally and iteratively. And also here, documentation is key. In a future blog post, we will offer more insights into the setup and results of our experimentation.  <h2>Search as a Service, Reloaded</h2> Content Graph can deliver content to you. It offers an intuitive query language that allows you to precisely filter, select and navigate the information you need based on strongly typed schemas. What you request is exactly what you get. Content delivery for building or populating sites is made simple and easy. But the core of Content Graph is driven by a search engine. Search engines are information systems designed to help find stored information with a query, which are often delivered as a list of ordered results. The primary advantage of a search engine is flexible retrieval of results with high performance in both efficiency and effectiveness. So a key differentiator of Content Graph are its search capabilities. We offer precise and powerful text matching. You can increase visitor engagement on your site by offering site search, e.g., adding a search box and facets, as well as different ways to rank your results very effectively, but also accurately and efficiently. We have value-based ordering by fields and have state-of-the-art relevance ranking. The information will be optimally returned in the order that will drive improved conversions. Great way to distribute content with visitors, but also between systems like web crawlers. We will continue offering new search capabilities that will make a big difference for our customers, and improve and go beyond what we offer now. And if you have ideas or a wishlist, you are <a href="https://feedback.optimizely.com/?project=PGQL">very welcome to share</a> them with us. I would argue that Content Graph is in essence primarily a search engine with graph capabilities --- but one that is offered as-a-service and comes out of the box without any complicated configuration needed, required steep learning curve and know-how, or operational costs. Building a search engine with your data has become very easy to provide access to your content. The time to market is reduced. And we allow you to focus on what’s most important to you: delivering optimal user experiences, which includes search experience, with content creation on your platform. Stay tuned for a future blog post by me about this topic. So, buckle up, we are in for a fun ride! The journey continues. <h2>Beta Availability</h2> Content Graph is available for all Optimizely DXP customers. Feel free to <a href="https://docs.developers.optimizely.com/digital-experience-platform/v1.4.0-content-graph/docs/getting-started">get started</a>. Help us gathering bugs! Users of the beta release are encouraged to report any bugs at <a href="https://www.optimizely.com/support/?_gl=1*14bemx2*_ga*NDA0ODc3OTY2LjE2NTEyMjQzMjg.*_ga_C7SLJ6HMJ5*MTY2MjU2NTE2MC4xMi4xLjE2NjI1NjczOTAuNTIuMC4w">our support team</a>. Feature or change requests are warmly welcomed as well. You can create ideas and feedback <a href="https://feedback.optimizely.com/?project=PGQL">here</a>.  The developers of Content Graph look forward to receiving your feedback on this beta release!2022-09-08T08:55:54.0000000Z

Blog post