Example - Airbnb financial data searchĀ¶
The code below is an example of hybrid search, a search algorithm that combines FTS and vector search in LanceDB.
Let's get stared with an example. In this notebook we'll use Airbnb financial data documents to search for "the specific reasons for higher operating costs" in a particular year.
# Setup
!pip install lancedb pandas langchain langchain-community pypdf openai cohere tiktoken sentence_transformers tantivy==0.20.1
import os
import getpass
# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = getpass.getpass()
Ā·Ā·Ā·Ā·Ā·Ā·Ā·Ā·
def pretty_print(docs):
for doc in docs:
print(doc + "\n\n")
from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
# Load $ABNB's financial report. This may take 1-2 minutes since the PDF is large
sec_filing_pdf = "https://d18rn0p25nwr6d.cloudfront.net/CIK-0001559720/8a9ebed0-815a-469a-87eb-1767d21d8cec.pdf"
# Create your PDF loader
loader = PyPDFLoader(sec_filing_pdf)
# Load the PDF document
documents = loader.load()
# Chunk the financial report
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
from langchain_community.vectorstores import LanceDB
from langchain.embeddings.openai import OpenAIEmbeddings
import lancedb
from lancedb.embeddings import get_registry
from lancedb.pydantic import Vector, LanceModel
openai = get_registry().get("openai").create()
class Schema(LanceModel):
text: str = openai.SourceField()
vector: Vector(openai.ndims()) = openai.VectorField()
embedding_function = OpenAIEmbeddings()
db = lancedb.connect("~/langchain")
table = db.create_table(
"airbnb",
schema=Schema,
mode="overwrite",
)
# Load the document into LanceDB
db = LanceDB.from_documents(docs, embedding_function, connection=table)
[2024-02-12T20:00:04Z WARN lance::dataset] No existing dataset at /Users/ayushchaurasia/langchain/airbnb.lance, it will be created
table.create_fts_index("text")
table.to_pandas().head()
text | vector | |
---|---|---|
0 | Table of Contents\nUNITED STATES\nSECURITIES A... | [-0.003405824, -0.03212391, 0.012812538, -0.02... |
1 | Class A common stock, par value $0.0001 per sh... | [-0.019193485, -0.02273649, 0.009623382, -0.02... |
2 | this chapter) during the preceding 12 months (... | [-0.020692078, -0.016187502, -0.008877442, -0.... |
3 | Indicate by check mark whether the registrant ... | [-0.019304628, -0.0034501317, -0.011525051, -0... |
4 | As of June 30, 2022, the aggregate market valu... | [-0.014594535, -0.011274607, -0.007967828, -0.... |
Vector SearchĀ¶
avg latency - 3.48 ms Ā± 71.6 Āµs per loop (mean Ā± std. dev. of 7 runs, 100 loops each)
query = "What are the specific factors contributing to Airbnb's increased operational expenses in the last fiscal year?"
docs = table.search(query).limit(5).to_pandas()["text"].to_list()
pretty_print(docs)
In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to the changes in the fair value of contingent consideration related to a 2019 acquisition. General and Administrative 2021 2022 % Change (in millions, except percentages) General and administrative $ 836 $ 950 14 % Percentage of revenue 14 % 11 % General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3 Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially adversely affected by a number of factors discussed elsewhere in these āRisk Factors,ā including: ā¢events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-related events, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other climate change on seasonal destinations; ā¢political, social, or economic instability; ā¢Hosts failing to meet guestsā expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic; Table of Contents Airbnb, Inc. Consolidated Statements of Operations (in millions, except per share amounts) Year Ended December 31, 2020 2021 2022 Revenue $ 3,378 $ 5,992 $ 8,399 Costs and expenses: Cost of revenue 876 1,156 1,499 Operations and support 878 847 1,041 Product development 2,753 1,425 1,502 Sales and marketing 1,175 1,186 1,516 General and administrative 1,135 836 950 Restructuring charges 151 113 89 Total costs and expenses 6,968 5,563 6,597 Income (loss) from operations (3,590) 429 1,802 Interest income 27 13 186 Interest expense (172) (438) (24) Other income (expense), net (947) (304) 25 Income (loss) before income taxes (4,682) (300) 1,989 Provision for (benefit from) income taxes (97) 52 96 Net income (loss) $ (4,585)$ (352)$ 1,893 Net income (loss) per share attributable to Class A and Class B common stockholders: Basic $ (16.12)$ (0.57)$ 2.97 Diluted $ (16.12)$ (0.57)$ 2.79 Our future revenue growth depends on the growth of supply and demand for listings on our platform, and our business is affected by general economic and business conditions worldwide as well as trends in the global travel and hospitality industries and the short and long-term accommodation regulatory landscape. In addition, we believe that our revenue growth depends upon a number of factors, including: ā¢global macroeconomic conditions, including inflation and rising interest rates and recessionary concerns; ā¢our ability to retain and grow the number of guests and Nights and Experiences Booked; ā¢our ability to retain and grow the number of Hosts and the number of available listings on our platform; ā¢events beyond our control such as pandemics and other health concerns, restrictions on travel and immigration, political, social or economic instability, including international
Hybrid SearchĀ¶
LanceDB support hybrid search with custom Rerankers. Here's the summary of latency numbers of some of the Reranking methods available
Let us now perform hybrid search by combining vector and FTS search results. First, we'll cover the default Reranker.
Linear Combination RerankerĀ¶
LinearCombinationReranker(weight=0.7)
is used as the default reranker for reranking the hybrid search results if the reranker isn't specified explicitly.
The weight
param controls the weightage provided to vector search score. The weight of 1-weight
is applied to FTS scores when reranking.
Latency - 71 ms Ā± 25.4 Āµs per loop (mean Ā± std. dev. of 7 runs, 100 loops each)
docs = table.search(query, query_type="hybrid").limit(5).to_pandas()["text"].to_list()
pretty_print(docs)
In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful (a) The Borrower may, at its election, deliver a Pricing Certificate to the Administrative Agent in respect of the most recently ended fiscal year, commencing with the fiscal year ended December 31, 2022, on any date prior to the date that is 270 days following the last day of such fiscal year (the -50- āInitial Delivery Dateā); provided that the Pricing Certificate for any fiscal year may be delivered on any date following the Initial Delivery Date that is prior to the date that is 365 days following the last day of the preceding fiscal year, so long as such Pricing Certificate includes a certification that delivery of such Pricing Certificate on or before the Initial Delivery Date was not possible because (i) the information required to calculate the KPI Metrics for such preceding fiscal year was not available at such time or (ii) the report of the KPI Metrics Auditor, if relevant, was not available at such time (the date of the Administrative Agentās receipt thereof, each a āPricing Certificate Dateā). Upon delivery of a Pricing Certificate in respect of a fiscal year, (i) the Applicable Rate for the Loans incurred by the Borrower shall be increased or decreased (or neither increased nor decreased), as applicable, pursuant to the Sustainability Margin Adjustment as set forth in the KPI Metrics including such Sustainability Pricing Adjustment Date and ending on the date immediately preceding the next Sustainability Pricing Adjustment Date. (b) For the avoidance of doubt, only one Pricing Certificate may be delivered in respect of any fiscal year. It is further understood and agreed that the Applicable Rate for Loans incurred by the Borrower will never be reduced or increased by more than 0.050% and that the Applicable Rate for the Revolving Commitment Fee will never be reduced or increased by more than 0.010%, pursuant to the Sustainability Margin Adjustment and the Sustainability Fee Adjustment, respectively, on any Sustainability Pricing Adjustment Date. For the avoidance of doubt, any adjustment to the Applicable Rate for such Loans or such Revolving Commitment Fee by reason of meeting one or both KPI Metrics in any fiscal year shall not be cumulative year-over-year. The adjustments pursuant to this Section made on any Sustainability Pricing Adjustment Date shall only apply for the period until the date immediately preceding the next Sustainability Pricing Adjustment Date. (c) If, for any fiscal year, either (i) no Pricing Certificate shall have been delivered for such fiscal year or (ii) the Pricing Certificate delivered for such fiscal year shall fail to include the Diverse Supplier Spend Percentage or GHG Emissions Intensity for such fiscal year, then the Sustainability Margin Adjustment will be positive 0.050% and/or the Sustainability Fee Adjustment will be positive 0.010%, as applicable, in each case commencing on the last day such Pricing Certificate could have been delivered in accordance with the terms of clause (a) above (it being understood that, in the case of the foregoing clause (ii), the Sustainability Margin Adjustment or the Sustainability Fee Adjustment will be determined in accordance with such Pricing Certificate to the extent the (A) Sustainability Margin
Cohere RerankerĀ¶
This uses Cohere's Reranking API to re-rank the results. It accepts the reranking model name as a parameter. By Default it uses the english-v3 model but you can easily switch to a multi-lingual model.
latency - 605 ms Ā± 78.1 ms per loop (mean Ā± std. dev. of 7 runs, 1 loop each)
# Free API key
os.environ["COHERE_API_KEY"] = getpass.getpass()
Ā·Ā·Ā·Ā·Ā·Ā·Ā·Ā·
from lancedb.rerankers import CohereReranker
reranker = CohereReranker()
docs = table.search(query, query_type="hybrid").limit(5).rerank(reranker=reranker).to_pandas()["text"].to_list()
pretty_print(docs)
Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition. In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined with sentiment of the workforce in China, and Chinaās policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to the changes in the fair value of contingent consideration related to a 2019 acquisition. General and Administrative 2021 2022 % Change (in millions, except percentages) General and administrative $ 836 $ 950 14 % Percentage of revenue 14 % 11 % General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3 Table of Contents Airbnb, Inc. Consolidated Statements of Operations (in millions, except per share amounts) Year Ended December 31, 2020 2021 2022 Revenue $ 3,378 $ 5,992 $ 8,399 Costs and expenses: Cost of revenue 876 1,156 1,499 Operations and support 878 847 1,041 Product development 2,753 1,425 1,502 Sales and marketing 1,175 1,186 1,516 General and administrative 1,135 836 950 Restructuring charges 151 113 89 Total costs and expenses 6,968 5,563 6,597 Income (loss) from operations (3,590) 429 1,802 Interest income 27 13 186 Interest expense (172) (438) (24) Other income (expense), net (947) (304) 25 Income (loss) before income taxes (4,682) (300) 1,989 Provision for (benefit from) income taxes (97) 52 96 Net income (loss) $ (4,585)$ (352)$ 1,893 Net income (loss) per share attributable to Class A and Class B common stockholders: Basic $ (16.12)$ (0.57)$ 2.97 Diluted $ (16.12)$ (0.57)$ 2.79 Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially adversely affected by a number of factors discussed elsewhere in these āRisk Factors,ā including: ā¢events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-related events, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other climate change on seasonal destinations; ā¢political, social, or economic instability; ā¢Hosts failing to meet guestsā expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic; In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful
Relevance score is returned by Cohere API and is independent of individual FTS and vector search scores.
table.search(query, query_type="hybrid").limit(5).rerank(reranker=reranker).to_pandas()
text | vector | _relevance_score | |
---|---|---|---|
0 | Increased operating expenses, decreased revenu... | [0.0034929817, -0.024774546, 0.012623285, -0.0... | 0.985328 |
1 | Made Possible by Hosts, Strangers, AirCover, C... | [-0.0042489874, -0.005382498, 0.007190078, -0.... | 0.979036 |
2 | Table of Contents\nAirbnb, Inc.\nConsolidated ... | [-0.008569201, -0.019810658, 0.014144964, -0.0... | 0.696578 |
3 | Our success depends significantly on existing ... | [0.0027109187, -0.028220002, 0.022864284, -0.0... | 0.539923 |
4 | In addition, the number of listings on Airbnb ... | [0.0068983347, -0.0147690065, 0.042441186, -0.... | 0.460713 |
ColBERT RerankerĀ¶
Colber Reranker is powered by ColBERT model. It runs locally using the huggingface implementation.
Latency - 950 ms Ā± 5.78 ms per loop (mean Ā± std. dev. of 7 runs, 1 loop each)
Note: First query might be slow. It is recommended to reuse the Reranker
objects as the models are cached. Subsequent runs will be faster on reusing the same reranker object
from lancedb.rerankers import ColbertReranker
reranker = ColbertReranker()
docs = table.search(query, query_type="hybrid").limit(5).rerank(reranker=reranker).to_pandas()["text"].to_list()
pretty_print(docs)
Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to the changes in the fair value of contingent consideration related to a 2019 acquisition. General and Administrative 2021 2022 % Change (in millions, except percentages) General and administrative $ 836 $ 950 14 % Percentage of revenue 14 % 11 % General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3 Our future revenue growth depends on the growth of supply and demand for listings on our platform, and our business is affected by general economic and business conditions worldwide as well as trends in the global travel and hospitality industries and the short and long-term accommodation regulatory landscape. In addition, we believe that our revenue growth depends upon a number of factors, including: ā¢global macroeconomic conditions, including inflation and rising interest rates and recessionary concerns; ā¢our ability to retain and grow the number of guests and Nights and Experiences Booked; ā¢our ability to retain and grow the number of Hosts and the number of available listings on our platform; ā¢events beyond our control such as pandemics and other health concerns, restrictions on travel and immigration, political, social or economic instability, including international Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially adversely affected by a number of factors discussed elsewhere in these āRisk Factors,ā including: ā¢events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-related events, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other climate change on seasonal destinations; ā¢political, social, or economic instability; ā¢Hosts failing to meet guestsā expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic; In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful Table of Contents Airbnb, Inc. Consolidated Statements of Operations (in millions, except per share amounts) Year Ended December 31, 2020 2021 2022 Revenue $ 3,378 $ 5,992 $ 8,399 Costs and expenses: Cost of revenue 876 1,156 1,499 Operations and support 878 847 1,041 Product development 2,753 1,425 1,502 Sales and marketing 1,175 1,186 1,516 General and administrative 1,135 836 950 Restructuring charges 151 113 89 Total costs and expenses 6,968 5,563 6,597 Income (loss) from operations (3,590) 429 1,802 Interest income 27 13 186 Interest expense (172) (438) (24) Other income (expense), net (947) (304) 25 Income (loss) before income taxes (4,682) (300) 1,989 Provision for (benefit from) income taxes (97) 52 96 Net income (loss) $ (4,585)$ (352)$ 1,893 Net income (loss) per share attributable to Class A and Class B common stockholders: Basic $ (16.12)$ (0.57)$ 2.97 Diluted $ (16.12)$ (0.57)$ 2.79
Cross Encoder RerankerĀ¶
Uses cross encoder models are rerankers. Uses sentence transformer implemntation locally
Latency - 1.38 s Ā± 64.6 ms per loop (mean Ā± std. dev. of 7 runs, 1 loop each)
from lancedb.rerankers import CrossEncoderReranker
reranker=CrossEncoderReranker()
docs = table.search(query, query_type="hybrid").limit(5).rerank(reranker=reranker).to_pandas()["text"].to_list()
pretty_print(docs)
Table of Contents Airbnb, Inc. Consolidated Statements of Operations (in millions, except per share amounts) Year Ended December 31, 2020 2021 2022 Revenue $ 3,378 $ 5,992 $ 8,399 Costs and expenses: Cost of revenue 876 1,156 1,499 Operations and support 878 847 1,041 Product development 2,753 1,425 1,502 Sales and marketing 1,175 1,186 1,516 General and administrative 1,135 836 950 Restructuring charges 151 113 89 Total costs and expenses 6,968 5,563 6,597 Income (loss) from operations (3,590) 429 1,802 Interest income 27 13 186 Interest expense (172) (438) (24) Other income (expense), net (947) (304) 25 Income (loss) before income taxes (4,682) (300) 1,989 Provision for (benefit from) income taxes (97) 52 96 Net income (loss) $ (4,585)$ (352)$ 1,893 Net income (loss) per share attributable to Class A and Class B common stockholders: Basic $ (16.12)$ (0.57)$ 2.97 Diluted $ (16.12)$ (0.57)$ 2.79 Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to the changes in the fair value of contingent consideration related to a 2019 acquisition. General and Administrative 2021 2022 % Change (in millions, except percentages) General and administrative $ 836 $ 950 14 % Percentage of revenue 14 % 11 % General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3 Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition. In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined with sentiment of the workforce in China, and Chinaās policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful Our future revenue growth depends on the growth of supply and demand for listings on our platform, and our business is affected by general economic and business conditions worldwide as well as trends in the global travel and hospitality industries and the short and long-term accommodation regulatory landscape. In addition, we believe that our revenue growth depends upon a number of factors, including: ā¢global macroeconomic conditions, including inflation and rising interest rates and recessionary concerns; ā¢our ability to retain and grow the number of guests and Nights and Experiences Booked; ā¢our ability to retain and grow the number of Hosts and the number of available listings on our platform; ā¢events beyond our control such as pandemics and other health concerns, restrictions on travel and immigration, political, social or economic instability, including international
(Experimental) OpenAI RerankerĀ¶
This prompts chat model to rerank results which is not a dedicated reranker model. This should be treated as experimental. You might run out of token limit so set the search limits based on your token limit.
NOTE: It is recommended to use gpt-4-turbo-preview
, older models might lead to bad behaviour
Latency - Can take 10s of seconds if using GPT-4 model
from lancedb.rerankers import OpenaiReranker
reranker=OpenaiReranker(model_name="gpt-4-turbo-preview")
docs = table.search(query, query_type="hybrid").limit(5).rerank(reranker=reranker).to_pandas()["text"].to_list()
pretty_print(docs)
Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to the changes in the fair value of contingent consideration related to a 2019 acquisition. General and Administrative 2021 2022 % Change (in millions, except percentages) General and administrative $ 836 $ 950 14 % Percentage of revenue 14 % 11 % General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3 Table of Contents Airbnb, Inc. Consolidated Statements of Operations (in millions, except per share amounts) Year Ended December 31, 2020 2021 2022 Revenue $ 3,378 $ 5,992 $ 8,399 Costs and expenses: Cost of revenue 876 1,156 1,499 Operations and support 878 847 1,041 Product development 2,753 1,425 1,502 Sales and marketing 1,175 1,186 1,516 General and administrative 1,135 836 950 Restructuring charges 151 113 89 Total costs and expenses 6,968 5,563 6,597 Income (loss) from operations (-3,590) 429 1,802 Interest income 27 13 186 Interest expense (-172) (-438) (-24) Other income (expense), net (-947) (-304) 25 Income (loss) before income taxes (-4,682) (-300) 1,989 Provision for (benefit from) income taxes (-97) 52 96 Net income (loss) $ (-4,585)$ (-352)$ 1,893 Net income (loss) per share attributable to Class A and Class B common stockholders: Basic $ (-16.12)$ (-0.57)$ 2.97 Diluted $ (-16.12)$ (-0.57)$ 2.79 In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially adversely affected by a number of factors discussed elsewhere in these āRisk Factors,ā including: ā¢events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-related events, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other climate change on seasonal destinations; ā¢political, social, or economic instability; ā¢Hosts failing to meet guestsā expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic; Our future revenue growth depends on the growth of supply and demand for listings on our platform, and our business is affected by general economic and business conditions worldwide as well as trends in the global travel and hospitality industries and the short and long-term accommodation regulatory landscape. In addition, we believe that our revenue growth depends upon a number of factors, including: ā¢global macroeconomic conditions, including inflation and rising interest rates and recessionary concerns; ā¢our ability to retain and grow the number of guests and Nights and Experiences Booked; ā¢our ability to retain and grow the number of Hosts and the number of available listings on our platform; ā¢events beyond our control such as pandemics and other health concerns, restrictions on travel and immigration, political, social or economic instability, including international
Use your custom RerankerĀ¶
Hybrid search in LanceDB is designed to be very flexible. You can easily plug in your own Re-reranking logic. To do so, you simply need to implement the base Reranker class
from lancedb.rerankers import Reranker
import pyarrow as pa
class MyCustomReranker(Reranker):
def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table:
combined_results = self.merge(vector_results, fts_results) # Or custom merge algo
# Custom Reranking logic here
return combined_results
Custom Reranker based on CohereRerankerĀ¶
For the sake of simplicity let's build custom reranker that just enchances the Cohere Reranker by accepting a filter query, and accept other CohereReranker params as kwags.
For this toy example let's say we want to get rid of docs that represent a table of contents, appendix etc. as these are semantically close of representing costs but this isn't something we are interested in because they don't represent the specific reasons why operating costs were high. They simply represent the costs.
from typing import List, Union
import pandas as pd
from lancedb.rerankers import CohereReranker
class MofidifiedCohereReranker(CohereReranker):
def __init__(self, filters: Union[str, List[str]], **kwargs):
super().__init__(**kwargs)
filters = filters if isinstance(filters, list) else [filters]
self.filters = filters
def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table:
combined_result = super().rerank_hybrid(query, vector_results, fts_results)
df = combined_result.to_pandas()
for filter in self.filters:
df = df.query("not text.str.contains(@filter)")
return pa.Table.from_pandas(df)
reranker = MofidifiedCohereReranker(filters="Table of Contents")
docs = table.search(query, query_type="hybrid").limit(5).rerank(reranker=reranker).to_pandas()["text"].to_list()
pretty_print(docs)
Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition. In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined with sentiment of the workforce in China, and Chinaās policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to the changes in the fair value of contingent consideration related to a 2019 acquisition. General and Administrative 2021 2022 % Change (in millions, except percentages) General and administrative $ 836 $ 950 14 % Percentage of revenue 14 % 11 % General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3 Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially adversely affected by a number of factors discussed elsewhere in these āRisk Factors,ā including: ā¢events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-related events, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other climate change on seasonal destinations; ā¢political, social, or economic instability; ā¢Hosts failing to meet guestsā expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic; In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatened enforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful Our future revenue growth depends on the growth of supply and demand for listings on our platform, and our business is affected by general economic and business conditions worldwide as well as trends in the global travel and hospitality industries and the short and long-term accommodation regulatory landscape. In addition, we believe that our revenue growth depends upon a number of factors, including: ā¢global macroeconomic conditions, including inflation and rising interest rates and recessionary concerns; ā¢our ability to retain and grow the number of guests and Nights and Experiences Booked; ā¢our ability to retain and grow the number of Hosts and the number of available listings on our platform; ā¢events beyond our control such as pandemics and other health concerns, restrictions on travel and immigration, political, social or economic instability, including international
As you can see the document containing the Table of contetnts of spending no longer shows up