diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 5ee1dba..3d691cc 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -1,3 +1,192 @@ +* xref:hello-world:overview.adoc[Couchbase {name_platform} SDK Overview] + +* xref:hello-world:start-using-sdk.adoc[Hello World] +** xref:hello-world:sample-application.adoc[] +** xref:hello-world:student-record-developer-tutorial.adoc[Student Record Tutorial] +** xref:howtos:managing-connections.adoc[Connecting to Your Database] +*** xref:howtos:sdk-authentication.adoc[Authentication & Authorization] +*** xref:howtos:troubleshooting-cloud-connections.adoc[Troubleshooting Cloud Connections] + +* xref:concept-docs:data-durability-acid-transactions.adoc[Data Modelling, Durability, and Consistency] +** xref:howtos:kv-operations.adoc[] +*** xref:howtos:caching-example.adoc +** xref:howtos:concurrent-document-mutations.adoc[] +** xref:howtos:subdocument-operations.adoc[] +** xref:concept-docs:data-model.adoc[] +*** xref:concept-docs:documents.adoc[] +// *** xref:concept-docs:collections.adoc[] +*** xref:howtos:json.adoc[Choosing a JSON Library] +// only Java and Scala have ^ +// Scala's we keep - Java's turns into the concept?? +*** xref:concept-docs:subdocument-operations.adoc[Sub-Documents] +*** xref:concept-docs:nonjson.adoc[] +*** xref:howtos:transcoders-nonjson.adoc[Transcoders & Non-JSON Documents] +*** xref:concept-docs:xattr.adoc[XATTR & Virtual XATTR] +** xref:howtos:encrypting-using-sdk.adoc[Encrypting Your Data] +*** xref:concept-docs:encryption.adoc[Understanding Field Level Encryption] +** xref:concept-docs:compression.adoc[] + +* xref:concept-docs:querying-your-data.adoc[] +** xref:howtos:sqlpp-queries-with-sdk.adoc[] +*** xref:concept-docs:n1ql-query.adoc[Understanding {sqlpp} Queries] +** xref:howtos:vector-searching-with-sdk.adoc[] +** xref:concept-docs:full-text-search-overview.adoc[Understanding Search] +*** xref:howtos:full-text-searching-with-sdk.adoc[Using the Search Service] +** xref:howtos:analytics-using-sdk.adoc[] + +* xref:howtos:distributed-acid-transactions-from-the-sdk.adoc[Distributed ACID Transactions] +** xref:howtos:transactions-single-query.adoc[] +** xref:howtos:transactions-tracing.adoc[] +** xref:concept-docs:transactions.adoc[] +*** xref:concept-docs:transactions-cleanup.adoc[] +*** xref:concept-docs:transactions-error-handling.adoc[] + +* xref:concept-docs:durability-replication-failure-considerations.adoc[] +** xref:howtos:collecting-information-and-logging.adoc[] +** xref:howtos:health-check.adoc[Using Health Check] +** xref:concept-docs:response-time-observability.adoc[Observability] +*** xref:howtos:slow-operations-logging.adoc[] +*** xref:howtos:observability-metrics.adoc[] +*** xref:howtos:observability-orphan-logger.adoc[] +*** xref:howtos:observability-tracing.adoc[] + +* xref:concept-docs:best-practices.adoc[] +** xref:howtos:concurrent-async-apis.adoc[Async & Reactive APIs] +** xref:howtos:error-handling.adoc[] +// ** xref:howtos:unit-testing.adoc[] + +* xref:concept-docs:management-api.adoc[Managing Couchbase] +** xref:howtos:provisioning-cluster-resources.adoc[] +** xref:howtos:sdk-user-management-example.adoc[Managing Users] + +* xref:project-docs:deployment.adoc[SDK Deployment] +** xref:project-docs:sdk-release-notes.adoc[Release Notes] +*** https://docs-archive.couchbase.com/home/index.html[Older Versions Archive^] +** xref:project-docs:compatibility.adoc[] +*** xref:project-docs:migrating-sdk-code-to-3.n.adoc[] +*** xref:project-docs:distributed-acid-transactions-migration-guide.adoc[] +*** xref:project-docs:third-party-integrations.adoc[] +** xref:project-docs:sdk-full-installation.adoc[] +*** xref:hello-world:platform-help.adoc[] +** xref:project-docs:get-involved.adoc[] +*** xref:project-docs:sdk-licenses.adoc[] +*** xref:home:contribute:index.adoc[Improve the Docs] + +* xref:ref:index.adoc[] +** https://docs.couchbase.com/sdk-api/couchbase-python-client[API Reference^] +** https://github.com/couchbase/couchbase-python-client[Python SDK source code^] +** xref:ref:client-settings.adoc[] +** xref:ref:error-codes.adoc[] +** xref:ref:glossary.adoc[Glossary] +** xref:ref:travel-app-data-model.adoc[] +** xref:project-docs:metadoc-about-these-sdk-docs.adoc[About These Docs] + + + + + + + + + +//// +.Getting Started +* xref:hello-world:start-using-sdk.adoc[] +* xref:howtos:kv-operations.adoc[] +* xref:howtos:n1ql-queries-with-sdk.adoc[] +* xref:howtos:full-text-searching-with-sdk.adoc[] +* xref:hello-world:sample-application.adoc[] +* xref:hello-world:spring-data-sample-application.adoc[] + +.Transactions +* xref:howtos:distributed-acid-transactions-from-the-sdk.adoc[] +** xref:howtos:transactions-single-query.adoc[] +** xref:howtos:transactions-tracing.adoc[] +* xref:concept-docs:transactions.adoc[] +** xref:concept-docs:transactions-cleanup.adoc[] +** xref:concept-docs:transactions-error-handling.adoc[] + +.Further Data Ops +* xref:howtos:json.adoc[] +* xref:howtos:subdocument-operations.adoc[] +// ** xref:howtos:sdk-xattr-example.adoc[Extended Attributes] +* xref:howtos:analytics-using-sdk.adoc[] +// ** xref:howtos:advanced-analytics-querying.adoc[Advanced Analytics Querying] +* xref:howtos:view-queries-with-sdk.adoc[] +* xref:howtos:concurrent-async-apis.adoc[Async & Reactive APIs] +* xref:howtos:concurrent-document-mutations.adoc[] +* xref:howtos:encrypting-using-sdk.adoc[] +* xref:howtos:transcoders-nonjson.adoc[Transcoders & Non-JSON Documents] + +.Managing Couchbase +* xref:howtos:managing-connections.adoc[] +** xref:howtos:sdk-authentication.adoc[] +** xref:howtos:troubleshooting-cloud-connections.adoc[Troubleshooting Cloud Connections] +* xref:howtos:provisioning-cluster-resources.adoc[] +** xref:howtos:sdk-user-management-example.adoc[] + +.Errors & Diagnostics +* xref:howtos:error-handling.adoc[] +* xref:howtos:collecting-information-and-logging.adoc[] +* xref:howtos:health-check.adoc[] +* Observability +** xref:howtos:slow-operations-logging.adoc[] +** xref:howtos:observability-metrics.adoc[] +** xref:howtos:observability-orphan-logger.adoc[] +** xref:howtos:observability-tracing.adoc[] + +.Learn +* xref:concept-docs:concepts.adoc[] +* xref:concept-docs:buckets-and-clusters.adoc[Buckets & Clusters] +* xref:concept-docs:collections.adoc[Collections & Scopes] +* xref:concept-docs:compression.adoc[] +* xref:concept-docs:data-model.adoc[] +** xref:concept-docs:documents.adoc[] +** xref:concept-docs:nonjson.adoc[] +** xref:concept-docs:subdocument-operations.adoc[Sub-Documents] +** xref:concept-docs:xattr.adoc[XATTR & Virtual XATTR] +* xref:concept-docs:errors.adoc[Errors & Diagnostics] +** xref:concept-docs:health-check.adoc[] +** xref:concept-docs:response-time-observability.adoc[] +* xref:concept-docs:durability-replication-failure-considerations.adoc[] +* xref:concept-docs:encryption.adoc[Field Level Encryption] +* xref:concept-docs:data-services.adoc[Service Selection] +** xref:concept-docs:analytics-for-sdk-users.adoc[] +** xref:concept-docs:understanding-views.adoc[Views] +** xref:concept-docs:n1ql-query.adoc[] +** xref:concept-docs:full-text-search-overview.adoc[Search] +* xref:concept-docs:sdk-user-management-overview.adoc[User Management] +** xref:concept-docs:certificate-based-authentication.adoc[Cert Auth] +** xref:concept-docs:rbac.adoc[RBAC] + +.References +* https://docs.couchbase.com/sdk-api/couchbase-java-client[API Reference] +* https://docs.couchbase.com/sdk-api/couchbase-core-io/[JVM Core IO API] +* https://docs.couchbase.com/sdk-api/couchbase-transactions-java/index.html?overview-summary.html[Transactions API Reference] +* xref:ref:client-settings.adoc[] +// * xref:ref:data-structures[Data Structures] +* xref:ref:error-codes.adoc[] +* xref:ref:glossary.adoc[Glossary] +* xref:ref:travel-app-data-model.adoc[] + +.Project Docs +* xref:project-docs:sdk-release-notes.adoc[SDK Release Notes] +* xref:project-docs:compatibility.adoc[] +** xref:project-docs:migrating-sdk-code-to-3.n.adoc[] +*** xref:howtos:working-with-collections.adoc[Working with Collections] +** xref:project-docs:distributed-acid-transactions-migration-guide.adoc[] +** xref:project-docs:third-party-integrations.adoc[] +* xref:project-docs:sdk-full-installation.adoc[] +// ** xref:hello-world:platform-help.adoc[] +* https://docs-archive.couchbase.com/home/index.html[Older Versions Archive] +* xref:project-docs:sdk-licenses.adoc[] +* xref:project-docs:get-involved.adoc[] + ** https://docs.couchbase.com/home/contribute/index.html[Improve the Docs] +* xref:project-docs:metadoc-about-these-sdk-docs.adoc[About These Docs] +//// + + +//// .Couchbase Python SDK .Getting Started @@ -91,3 +280,4 @@ * xref:project-docs:get-involved.adoc[Get involved] ** https://docs.couchbase.com/home/contribute/index.html[Improve the Docs] * xref:project-docs:metadoc-about-these-sdk-docs.adoc[About These Docs] +//// diff --git a/modules/concept-docs/pages/data-durability-acid-transactions.adoc b/modules/concept-docs/pages/data-durability-acid-transactions.adoc new file mode 100644 index 0000000..18cefe6 --- /dev/null +++ b/modules/concept-docs/pages/data-durability-acid-transactions.adoc @@ -0,0 +1,375 @@ += Data Modelling, Durability, and Consistency +// :page-aliases: durability-replication-failure-considerations.adoc +:description: Performance, availability, consistency -- balance your priorities, and model your data to achieve these goals. +:page-toclevels: 2 +:anyreplica-api: pass:q[https://docs.couchbase.com/sdk-api/couchbase-python-client/couchbase_api/couchbase_core.html#couchbase.collection.Collection.get_any_replica] + + +// Note to editors +// +// This page pulls in content from -sdk-common- +// and code samples from -example-dir- +// +// It can be seen built at http://docs.couchbase.com/python-sdk/4.5/concept-docs/data-durability-acid-transactions.html + + +[abstract] +{description} + + +You want to query your data, but does that always mean a {sqlpp} query to the Query Service? +The CRUD API of the {sqlpp} Query service is mirrored by the Data Service's API, but without the need for indexes, and using a binary protocol gives faster access than streaming JSON from the Query Service. + + +[source,python] +---- +coll.upsert('key', 'value') +get_res = coll.get("key") +print('Get result - value: {}; CAS: {}'.format(get_res.content, get_res.cas)) +---- + +At its heart, a database is in the business of storing data and letting you query and retrieve it. +Putting a database on the network -- in particular a remote network, or another company's cloud service -- and partitioning the data across multiple nodes, with several replicas, does not alter this. +It does, however, mean that choices must be made to optimize for consistency or availability of data. + +.Skipping Ahead +[TIP] +==== +This page lays out some of the things you need to consider when designing an app. +If you have already reached your decisions, and want to work with the Data API, then skip straight to our pages on +xref:howtos:kv-operations.adoc[], +xref:howtos:subdocument-operations.adoc[], or +xref:howtos:concurrent-document-mutations.adoc[], +and try some of the code snippets there. +Or see some of the other links in the <<#further-reading,Further Reading>> section. +==== + +Whether you go through the Data Service, or Query, +you'll find that both follow the typical DML (Data Manipulation Language) patterns that you encounter in the relational database world. +See the xref:querying-your-data.adoc[SDK Query introduction] for choices of {sqlpp} queries for OLTP (transactional queries) and OLAP (analytics) -- +including real-time analytics -- as well as fuzzy searches and vector search. + + + + +//// +// earlier doc intro: +Couchbase stores data in _documents_, and this is the atomic.... + +//// + +Designing an application to offer guarantees of durability or consistency in a networked environment is hard. +The newcomer to the area has to wrestle with concepts like https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed/[CAP Theorum^] and the https://en.wikipedia.org/wiki/Isolation_(database_systems)[many possible interpretations^] of ACID transactions. +Here we hope to keep it _relatively_ simple, with a look at which Couchbase features give you which guarantees, +when you may prefer to use them, and -- perhaps equally important, though sometimes overlooked -- +what is the performance cost of these choices. + + + +//// +From 3.4 doc: + +Data durability refers to the fault tolerance and persistence of data in the face of software or hardware failure. Even the most reliable software and hardware might fail at some point, and along with the failures, introduce a chance of data loss. Couchbase's durability features include Synchronous Replication, and the possibility to use distributed, multi-document ACID transactions. It is the responsibility of the development team and the software architect to evaluate the best choice for each use case. +Couchbase's distributed and scalable nature exposes any set-up to the risk of potential network and hardware problems. The key to durability is planning for resilience, by evaluating the options on offer for persistence and replication, and carefully considering the performance trade-offs involved. + +Durability +Writes in Couchbase (from the SDK or elsewhere) are written to a single node. From there, Couchbase Server will take care of sending the mutation to any configured replicas, and to disk. By default all writes are asynchronous, but levels of durability can be set, to ensure replication and/or persistence to disks, before the write is committed. + + +//// + + + + +== Data and Good Schema Design + +Each level of durability guarantee carries a price in time to persist to replicas or to disk. +Transactional guarantees across multiple documents add another layer of performance cost -- something readily borne when the data is important enough, and justifiably discarded for many categories of transient and unimportant data. +However, many data fall into an intermediate category, and here the atomic consistency decision is made by what's tolerable for read and write times. +In this case, consider if a tweak to the schema, to bring items into the same document, can give you transactional guarantees without the performance penalty of multi-document transactions. + +Consider, too, that many operations are performed at the _collection_ level, and keeping documents in the same collection can make for speedier indexing and queries -- whether {sqlpp} or Search. + +The Server enforces no schema, enabing evolutionary changes to your data model that reflect changes in the real world. +The schema-on-read approach allows the client software that you write with the SDK to work with changes to an implicit schema, +and allows heterogeneous data. + + +=== Objects, Relations, Tables + +In the Relational Database (RDBMS) world, a translation layer is often used between the objects of your data model in your application, +and the tables that you store the data in. +JSON storage allows you to store complex types, like nested records and arrays, without decomposing them to a second table +(known in the SQL world as https://en.wikipedia.org/wiki/Database_normalization[database normalization^]). + +When the relational model was proposed, more than 50 years ago, limitations in available computer resources meant that removing data duplication in one-to-many and many-to-many relationships this way made a great deal of sense. +There is still a case to be made for it for reducing inconsistencies -- the difference with a document database is that you get to choose when to do this. + + +=== Collections and Scopes + +Couchbase's atomic units of data are documents, stored as key-value pairs. +The value can be anything, but storing in JSON format enables indexing, searching, and many useful ways of working with the data from the SDK. + +Collections are arbitary groupings of the data documents. +Ones that suit your object model. +For example, one collection of students enrolled at the college +and one collection of courses available for them to take. +Notionally you may view them as equivalent to an RDBMS table -- but it's up to you. + +Within a bucket, you can organize your collections into scopes -- some methods are available at the bucket level, +but Search and Query Services favor Scope-level indexing and querying for greater efficiency. + + + + + + +// mention arrays where? +// x.rowsAs(JsonArray.class) + +// extra JSON choices - libs in Scala + + + +== Durability + +Writes in Couchbase (from the SDK or elsewhere) are written to a single node. +From there, Couchbase Server will take care of sending the mutation to any configured replicas, and to disk. +By default all writes are asynchronous, but levels of durability can be set, to ensure replication and/or _persistence_ to disks, before the write is _committed_. + + +// === Durable Writes + +include::{version-common}@sdk:shared:partial$durability-replication-failure-considerations.adoc[tag=syncrep] + +Further discussion can be found in the xref:concept-docs:durability-replication-failure-considerations.adoc[Durability and Failure Considerations documentation]. + +A practical look at this -- using the `durability` parameter with mutating operations -- can be found on the xref:howtos:kv-operations.adoc#durability[Data Service page]. + + + +This durability is enforced by the cluster. +Note, if you are working with an older, no longer supported version of Couchbase Server, you may want to look at +https://docs-archive.couchbase.com/python-sdk/3.3/concept-docs/durability-replication-failure-considerations.html#older-server-versions[client-verified durability in our archived docs]. + + + + + +== Reading from Replicas + +By default, read requests from the SDK are directed to the node with the active vbucket. +This bucket will usually be updated before any replicas (see <> above), +and is best for consistency. + +Some use cases benefit -- for consistency or availability -- from accessing both the active copy of the document, and/or one or all replicas. +This can be direct from the {anyreplica-api}[data service], +or a xref:howtos:subdocument-operations.adoc#reading-sub-documents-from-replicas[SubDoc Read from Replica]. +The method can either be one to return the first result returned -- active or replica -- +which is useful if a node is timing out; +or to return _all_ of the results, for client code to handle any inconsistency, or to build a consensus answer. + + +=== Preferred Server Group Replica Reads + +IMPORTANT: Preferred Server Group Replica Reads are only accessible with the {name-sdk} working with Couchbase Server 7.6.2 or newer (Capella or self-managed), from SDK version 4.4.0. + +xref:server:learn:clusters-and-availability/groups.adoc#understanding-server-group-awareness[Server Groups] +can be used to define subsets of nodes within a Couchbase cluster, which contain a complete set of vbuckets (active or replica). +As well as high availability use cases, Server Groups can also be used to keep much traffic within the same cloud Availability Zone. + +For Capella users with high data volumes, egress charges for reads from other Availability Zones (AZ) in AWS can be a significant cost. +The {name-sdk}, when making read replica requests, can make a request to a preferred Server Group -- +in this case the local AZ -- +and set to always read from a copy of the document in this local zone. +This is done by putting cluster nodes in the same AZ into the same +xref:server:learn:clusters-and-availability/groups.adoc#server-groups-and-vbuckets[Server Group], too. + +This may mean the application has to be tolerant of slight inconsistencies, +until the local replica catches up. +Alternatively, it may demand a stronger level of durability, to ensure that all copies of a document are consistent before they are accessible -- +provided that this is `persistToMajority` with xref:server:learn:data/durability.adoc#majority[no more than one replica]. + +Couchbase does not recommend this feature where read consistency is critical, +but with the appropriate durability settings consistency can be favored ahead of availability. + +[CAUTION] +.Replicas, Nodes, and Server Groups +==== +Implicit in the rules for durability, and the process of setting up Server Groups, is the following information -- which we mention here explicitly to ensure it is all noted: + +* Moving servers between Server Groups updates the `clustermap` immediately, but to move the data, an administrator *must* perform rebalance. Until the rebalance is complete, the SDK will see and be able to 'use' the new server groups, but the `vBucketMap` may still refer to data in the previous locations. + +* The cluster should have enough nodes and group to make sure that copies of the same document are not stored on the same node, and each group has nodes that cover all 1024 vbuckets (in other words, the number of the groups does not exceeds number of the copies: `active+num_replicas`). The Admin UI should emit small yellow warning if the configuration is considered unbalanced. + +* Setting *three* replicas for the bucket xref:server:learn:data/durability.adoc#majority[disables durability for sync writes], also precluding the use of +xref:concept-docs:transactions.adoc[multi-document ACID transactions]. +==== + + +// TODO Python code sample + + + +//// +.getReplicaFromPreferredServerGroup example, using Transactions +[source,java] +---- + cluster.transactions().run((ctx) -> { + TransactionGetResult getResult; + try { + getResult = ctx.getReplicaFromPreferredServerGroup(collection, id); + } catch (DocumentUnretrievableException err) { + getResult = ctx.get(collection, id); + } + }); +---- +//// + + + + + + +== Concurrent Document Mutations + +You can use the CAS (Compare and Swap) value to control how concurrent document modifications are handled. +It helps avoid and control potential race conditions in which some mutations may be inadvertently lost or overridden by mutations made by other clients. + +=== Performance considerations + +CAS operations incur no additional overhead. +CAS values are always returned from the server for each operation. +Comparing CAS at the server involves a simple integer comparison which incurs no overhead. + +=== CAS value format + +The CAS value should be treated as an opaque object at the application level. +No assumptions should be made with respect to how the value is changed (for example, it is wrong to assume that it is a simple counter value). +In the SDK, the CAS is represented as a 64 bit integer for efficient copying but should otherwise be treated as an opaque 8 byte buffer. + + + +== Pessimistic locking + +Pessimistic locking is the default approach for many RDBMS. +With Couchbase, optimistic locking with CAS is the recommended way to perform locking and concurrency control. + +Should you need the guarantees of explicit locking, documents can be locked using the _get-and-lock_ operation +and unlocked either explicitly using the _unlock_ operation +or implicitly by mutating the document with a valid CAS. +While a document is locked, it may be retrieved but not modified without using the correct CAS value. +When a locked document is retrieved, the server will return an invalid CAS value, preventing mutations of that document. + +More information, including a table of behaviors while an item is locked, can be found in the +xref:howtos:concurrent-document-mutations.adoc#pessimistic-locking[Pessimistic Locking section of the Concurrent Document Mutations page]. + + + + +== Expiration + +The expiration setting for a document determines if and when it expires. +When a document expires, Couchbase Server removes it. +You can set a maximum time to live (`max_expiry`) value on buckets and collections that imposes a default expiration on their documents. +It also imposes an upper limit on explicitly-set expiration times. + +You can set an expiration value on an individual document either when you create it or when you mutate it. +Set the expiration to the number of seconds the document should exist before Couchbase Server automatically removes it. +You can change this value later in case you want extend the life of the document. + +As there could be cost implications for storing ephemeral data past its usable life -- +or even legal implications about keeping data under GDPR and other privacy legislation -- +you should be clear about the interactions between setting `max_expiry` at different levels can have. + +For example, setting a collection's `max_expiry` to `0` (the default) means it will inherit the bucket's `maxExpiry` value; +setting a bucket's `max_expiry` to `0` (the default) means it will never expire. +The table in the xref:{version-server}@server:learn:data/expiration.adoc#expiration-setting-priorities[expiration documentation] +covers the interaction between a document’s expiration setting and the `max+expiry` settings of the collection and bucket that contain it. + +TIP: In earlier releases of the SDK, `maxTTL` was used for setting the time to live. +This method has been deprecated. +Please use `max_expiry` at the bucket and collection level. + +// TODO check they all use MAX32INT -- 2147483648 seconds + +=== Collection No-Expiry Option + +From Python SDK 4.1.11 with Capella (and self-managed Server from 7.6.0), setting a `max_expiry` of `-1` on a collection will prevent it from expiring, even when its containing bucket has reached expiry time. +Again, see the table in the xref:{version-server}@server:learn:data/expiration.adoc#expiration-setting-priorities[expiration documentation]. + +=== Changing a Document, but Not the Expiry + +By default, changing a document also changes its expiry. +If you do not pass a value for expiry, the document does not expire, even if the document previously had an expiry. +If this is not what you want, you must tell Couchbase to keep the document’s expiry. + +With the mutation operation on a document (say a `replace` or an `upsert`), use `preserve_expiry = true` -- +and consider using optimistic locking if two threads could be changing the same document and one is not necessarily passing in `preserve_expiry = true`. + + + + +== Multi-Document ACID Transactions + +include::{version-common}@sdk:shared:partial$acid-transactions.adoc[tags=intro] +See the xref:transactions.adoc[{name-sdk} transactions documentation]. + + +== Additional Information + + +=== Further Reading + + + + + +This section of the docs covers the Data Service -- Couchbase's key-value document store, the heart of the database -- from the client perspective. +It addresses: + +* Schemas for semi-structured data. +* The sub-document API for retrieving and modifying only a portion of a document (saving network bandwidth and transmission time). +* Durability options. +* Field-Level Encryption. +* Storing non-JSON documents. +* & Concurrent Document Mutations. + + + + + + + +Below, we link some of the pages in this section that take a deeper dive into data structures, durability guarantees, and avoiding race conditions during concurrent document modifications -- +but if you are in a hurry to just learn more about querying your data, please skip ahead to xref:concept-docs:querying-your-data.adoc[the next section]. + +* A deeper dive into our xref:server:learn:data/scopes-and-collections.adoc[scopes and collections documentation]. + +* xref:data-model.adoc#data-structures[Working with other data structures from the SDK]. + +* Backups + +* Retry strategy + + + +For more on Durability, +see xref:durability-replication-failure-considerations.adoc[]. + + + + + +=== Lowest Latency Streams + +Streaming in a distributed system is complex, and thus we do not make our internal streams and changes feed directly available. + +However, it is exposed through use of our xref:spark-connector::index.adoc[Spark] or xref:kafka-connector::index.adoc[Kafka] connectors, which give you a high level API to our low level primitives. + + +The source connector streams documents from Couchbase Server using the high-performance Database Change Protocol (DCP) and publishes the latest version of each document to a Kafka topic in near real-time. + +The sink connector subscribes to Kafka topics and writes the messages to Couchbase Server. + diff --git a/modules/howtos/examples/kv_operations.py b/modules/devguide/examples/python/kv_operations.py similarity index 100% rename from modules/howtos/examples/kv_operations.py rename to modules/devguide/examples/python/kv_operations.py diff --git a/modules/howtos/examples/n1ql_ops.py b/modules/devguide/examples/python/sqlpp_ops.py similarity index 100% rename from modules/howtos/examples/n1ql_ops.py rename to modules/devguide/examples/python/sqlpp_ops.py diff --git a/modules/hello-world/pages/overview.adoc b/modules/hello-world/pages/overview.adoc index 7502fad..fdce3fb 100644 --- a/modules/hello-world/pages/overview.adoc +++ b/modules/hello-world/pages/overview.adoc @@ -1,3 +1,212 @@ += Couchbase {name-sdk} {sdk_dot_minor} +:page-layout: landing-page-top-level-sdk +:page-role: tiles +:!sectids: + + +// Note to editors +// +// This page pulls in content from -sdk-common- +// and code samples from -example-dir- +// +// It can be seen built at wwww. +// +// See the antora.yml file for the {attributes} + + + += Couchbase {name-sdk} {sdk_dot_minor} + +The Couchbase {name-sdk} allows {name_platform} applications to access a Couchbase cluster -- +Capella or self-managed. +// -- other SDKs are available for Couchbase Lite (edge or mobile devices) and Capella Columnar (real-time analytics) + +xref:hello-world:start-using-sdk.adoc[Quickstart Guide] | +xref:project-docs:sdk-release-notes.adoc[] | +{sdk-api-link}[{name-sdk} API Reference] | +{sdk-gh-link}[{name-sdk} SDK source code] + + +What's the point of a fast and scalable database if it's not easy to develop for? +Couchbase gives you the {name_platform} APIs to work with Capella, our managed solution, or self-managed options in your private Cloud or datacenter. + + +[{tabs}] +==== +Data Ops (CRUD):: ++ +-- + +[source,python] +---- +include::devguide:example$python/kv_operations.py[indent=0,tag=durability] +---- +-- + +{sqlpp} Query (OLTP):: ++ +-- + +[source,python] +---- +include::devguide:example$python/sqlpp_ops.py[tag=named_options,indent=0] +---- +-- + +Vector Search:: ++ +-- + +[source,python] +---- +VectorQuery.create(field_name, vector, num_candidates=num_candidates, prefilter=MatchQuery('primary', field='color_wheel_pos')) +---- +-- +==== + + +Couchbase is a large platform -- covering many services -- and Couchbase SDKs are not thin wrappers generated around a REST API, but well thought out interfaces to the platform that make it easier to design and maintain your client code, +and work with Couchbase in more natural ways for your platform. +Install the SDK, and explore in the way that works best for you. + + +[source,console] +---- +$ python3 -m pip install couchbase +---- + + +//// +The Couchbase Python SDK integrates into the Python ecosystem through a number of extensions and connectors, including: + +* https://spring.io/projects/spring-data-couchbase[Spring Data] +* https://blog.couchbase.com/couchbase-spring-boot-spring-data/[Spring Boot] +* xref:quarkus-extension:ROOT:overview.adoc[Couchbase Quarkus Java Extension] +* xref:3.5@spark-connector:ROOT:java-api.adoc[Apache Spark Connector] +//// + +== Exploring the Python SDK + +The links in the sections below will take you where you want to go -- as will the navigation on the left-hand side of this page. +But if you don't know exactly where you need to go, try one of the following: + +* Our xref:hello-world:start-using-sdk.adoc[Quickstart Guide] introduces the SDK with a quick install, and CRUD examples against the Data Service. +* Couchbase's familiar SQL-family query language and fuzzy search options (including vector search) are introduced on the xref:concept-docs:querying-your-data.adoc[] page. +* The {name-sdk} docs are, necessarily, just a sub-set {sdk-api-link}[{name-sdk} API Reference] -- and a complete listing of all APIs can be found in the reference. +* For a fuller orientation, there is a xref:project-docs:metadoc-about-these-sdk-docs.adoc[guide to the {name-sdk} docs] + + +{empty} + + +== icon:database[] Using Your Database + +How-to guides to help you start your development journey with Couchbase and the {name-sdk}. + +++++ +
+++++ + +[.column] +.Easy to Connect & Get Started +* xref:hello-world:start-using-sdk.adoc[Getting Started] +* xref:hello-world:sample-application.adoc[] +* xref:hello-world:student-record-developer-tutorial.adoc[Beginners' Couchbase Tutorial] +* xref:howtos:managing-connections.adoc[] + +[.column] +.Search, Query, Analyze +* xref:howtos:sqlpp-queries-with-sdk.adoc[Query with a familiar, SQL-like language] +* xref:howtos:vector-searching-with-sdk.adoc[Vector Search for your AI app] +* xref:howtos:full-text-searching-with-sdk.adoc[Fuzzy Search with text and Geo data] +* xref:howtos:analytics-using-sdk.adoc[OLAP -- long running analytical queries] + +// For Real-Time Analytics, see our xref:[Capella Columnar SDKs]. + +[.column] +.Lightning Fast Data Service +* xref:howtos:kv-operations.adoc[] +* xref:howtos:subdocument-operations.adoc[] +* xref:howtos:encrypting-using-sdk.adoc[] +* xref:howtos:distributed-acid-transactions-from-the-sdk.adoc[Multi-Document Distributed ACID Transactions] + +[.column] +.Observability & Error Handling +* xref:howtos:error-handling.adoc[] +* xref:howtos:collecting-information-and-logging.adoc[] +* xref:howtos:slow-operations-logging.adoc[] +* xref:howtos:health-check.adoc[] + +++++ +
+++++ + +//// +== icon:graduation-cap[] Learn + +Take a deep-dive into the SDK concept material and learn more about Couchbase. + +++++ +
+++++ + +[.column] +.Data Concepts +* xref:concept-docs:data-model.adoc[] +* xref:concept-docs:data-services.adoc[Service Selection] +* xref:concept-docs:encryption.adoc[Field Level Encryption] + +[.column] +.Errors & Diagnostics Concepts +* xref:concept-docs:errors.adoc[] +* xref:concept-docs:response-time-observability.adoc[] +* xref:concept-docs:durability-replication-failure-considerations.adoc[] + +++++ +
+++++ +//// + +{empty} + + +== icon:book[] Resources + +Useful resources to help support your development experience with Couchbase and the {name_platform} SDK. + +++++ +
+++++ + +[.column] +.Reference +* {sdk-api-link}[API Reference^] +* xref:ref:client-settings.adoc[] +* xref:ref:error-codes.adoc[] +* {sdk-gh-link}[SDK source code] +// * xref:ref:glossary.adoc[Glossary] +// * xref:ref:travel-app-data-model.adoc[] + +[.column] +.Deployment +* xref:project-docs:sdk-release-notes.adoc[] +* xref:project-docs:compatibility.adoc[Compatibility] +// * https://docs-archive.couchbase.com/home/index.html[Older Versions Archive] +// ** xref:project-docs:migrating-sdk-code-to-3.n.adoc[] +* xref:project-docs:third-party-integrations.adoc[] +* xref:quarkus-extension:ROOT:overview.adoc[Couchbase Quarkus Java Extension] +* xref:project-docs:sdk-full-installation.adoc[] + +++++ +
+++++ + +include::{version-common}@sdk:shared:partial$columnar-available.adoc[tag=initial-availability] + + + + + + +//// = Couchbase Python SDK 4.5 :page-layout: landing-page-top-level-sdk :page-role: tiles @@ -126,3 +335,4 @@ Useful resources to help support your development experience with Couchbase and ++++ include::{version-common}@sdk:shared:partial$columnar-available.adoc[tag=initial-availability] +//// diff --git a/modules/hello-world/pages/start-using-sdk.adoc b/modules/hello-world/pages/start-using-sdk.adoc index e4d70a6..af736cd 100644 --- a/modules/hello-world/pages/start-using-sdk.adoc +++ b/modules/hello-world/pages/start-using-sdk.adoc @@ -1,3 +1,911 @@ += Hello World +:page-aliases: ROOT:getting-started.adoc,ROOT:start-using.adoc,ROOT:hello-couchbase.adoc,ROOT:start-using-sdk.adoc,ROOT:java-intro.adoc,ROOT:tutorial.adoc +:page-toclevels: 3 +:description: Install, connect, try. A quick start guide to get you up and running with Couchbase and the {name-sdk}. +:forum-link: https://www.couchbase.com/forums/c/java-sdk/5 +:page-partial: + +// Note to editors +// +// This page pulls in content from +// https://github.com/couchbase/docs-sdk-common/blob/release/8.0/modules/shared/partials/start-using-sdk.adoc +// +// and code samples from +// https://github.com/couchbase/docs-sdk-python/tree/release/4.5/modules/devguide/examples/python +// +// It can be seen built at +// https://docs.couchbase.com/python-sdk/4.5/hello-world/start-using-sdk.html + + +[abstract] +{description} + + + +Couchbase has a simple interface for creating and modifying records in a document, +based upon the *collection* into which the documents are organized. +You can read more about data modeling <>, +but first let's look at those data operations, and installing the {name-sdk}. + +// remember to add a nomenclature note to the next version of this page for LCB. + +.Upsert with Replication set to xref:concept-docs:data-durability-acid-transactions.adoc#durable-writes[Majority Durablity]: + +[source,python] +---- +include::devguide:example$python/kv_operations.py[indent=0,tag=durability] +---- + +`upsert` inserts (creates) the document if it does not exist, or replaces it if it does. +We'll explore creating and retrieving data records in more detail <> +(and touch lightly upon a little of Java's functional programming approach as we go), +after walking through a quick installation. + +TIP: This page walks you through a quick installation, and CRUD examples against the Data Service. +Elsewhere in this section you can find a fully worked-through xref:hello-world:sample-application.adoc[] and, +for those new to document (NoSQL) databases, our xref:hello-world:student-record-developer-tutorial.adoc[Student Record Tutorial]. + + +== Before You Start + +Couchbase Capella, our Database-as-a-Service, lets you get on with what matters, while we take care of the administration for you. +Alternately, if you need to control every aspect of deployment -- +or just want to run the Server in a VM on your laptop -- +there are several self-managed options available: + +[tabs] +==== +Couchbase Capella:: ++ +-- +If you haven't already got a cluster set up, the easiest route is to https://cloud.couchbase.com/sign-up[sign up to Couchbase Capella and deploy a free tier cluster^], +then come back to this page. +Make a note of the xref:cloud:get-started:connect.adoc[endpoint] to connect to, +and remember the credentials for the user that you set up. +-- + +Self-Managed Couchbase Server:: ++ +-- +Install Couchbase Server locally, or in your private Cloud: + +* xref:{version-server}@server:install:get-started.adoc[Deployment overview] +* xref:{version-server}@server:install:getting-started-docker.adoc[Docker Install] +* xref:operator::overview.adoc[Couchbase Autonomous Operator] +** xref:operator::install-kubernetes.adoc[Kubernetes] +** xref:operator::install-openshift.adoc[Openshift] +* xref:{version-server}server:cloud:couchbase-cloud-deployment.adoc[Cloud Marketplace]: +** xref:{version-server}@server:cloud:couchbase-aws-marketplace.adoc[AWS Marketplace] +** xref:{version-server}@server:cloud:couchbase-azure-marketplace.adoc[Azure Marketplace] +** xref:{version-server}@server:cloud:couchbase-gcp-cloud-launcher.adoc[GCP Marketplace] + +For the example code below to run, you'll need the username and password of the Administrator user that you create, and the IP address of at least one of the nodes of the cluster. +-- +==== + + +// Script the docs! +// Can any of the SDK installations be scripted? + +=== Prerequisites + +* The Python SDK is tested against LTS versions of Oracle JDK and OpenJDK -- +see the xref:project-docs:compatibility.adoc#jdk-compat[compatibility docs]. ++ +https://adoptium.net/[OpenJDK {java_latest_lts_version} with HotSpot JVM] is recommended. + +The code examples also assume: + +[tabs] +==== +Couchbase Capella:: ++ +-- +* You have signed up to https://cloud.couchbase.com/sign-up[Couchbase Capella]. + +* You have created your own bucket, or loaded the Travel Sample dataset. +Note, the Travel Sample dataset is installed automatically when deploying a Capella free tier cluster. + +* A user is created with permissions to access the cluster (at least Application Access permissions). +See the xref:cloud:get-started:cluster-and-data.adoc#credentials[Capella connection page] for more details. + +IMPORTANT: Couchbase Capella uses xref:cloud:organizations:organization-projects-overview.adoc[Roles] to control user access to cluster resources. +For the purposes of this guide, you can use the *Organization Owner* role automatically assigned to your account during installation of the Capella cluster. +In production, Couchbase strongly recommends setting up users with more granular access roles as a best practice for data security. +-- + +Self-Managed Couchbase Server:: ++ +-- +* xref:{version-server}@server:getting-started/do-a-quick-install.adoc[Couchbase Server] is installed and accessible locally. + +* You have created your own bucket, or loaded the Travel Sample dataset using the xref:{version-server}@server:manage:manage-settings/install-sample-buckets.adoc#install-sample-buckets-with-the-ui[Web interface]. + +* A user is created with permissions to access your cluster (at least Application Access permissions). +See xref:{version-server}@server:manage:manage-security/manage-users-and-roles.adoc[Manage Users, Groups and Roles] for more details. + +IMPORTANT: Couchbase Server uses xref:{version-server}@server:learn:security/roles.adoc[Role-Based Access Control (RBAC)] to control access to cluster resources. +In this guide we suggest using the *Full Admin* role created during setup of your local Couchbase Server cluster. +In production, Couchbase strongly recommends setting up users with more granular access roles as a best practice for data security. +-- +==== + + +== Installation + +We recommend running the latest Java LTS version (i.e. at the time of writing JDK {java_latest_lts_version}) with the highest patch version available. +// Other supported Java versions will work, too. +Couchbase publishes all stable artifacts to https://central.sonatype.com/namespace/com.couchbase.client[Maven Central]. + +The latest version of {sdk_dot_minor}.x is https://central.sonatype.com/artifact/com.couchbase.client/java-client/{sdk_current_version}/jar[{sdk_current_version}]. + +// tag::quick-install[] +More details of the installation process are in the +xref:project-docs:sdk-full-installation.adoc[full installation guide]. +In most cases, given the above prerequisites, use your favorite dependency management tool to install the SDK. + +[{tabs}] +==== +Maven:: ++ +-- +[source,xml,subs="+attributes"] +---- + + + com.couchbase.client + java-client + {sdk_current_version} + + +---- +-- + +Gradle:: ++ +-- +[source,groovy,subs="+attributes"] +---- +implementation 'com.couchbase.client:java-client:{sdk_current_version}' +---- +-- +==== +// end::quick-install[] + +=== IDE Plugins + +To make development easier, Couchbase plugins are available for VSCode and the IntelliJ family of IDEs and editors. +For links and more information on these and other integrations across the {name_platform} ecosystem, +check out the xref:project-docs:third-party-integrations.adoc[] page. + + +=== Grab the Code + +If you're all set up and in a real hurry, just grab this code sample and add in your Capella details. + +.Complete Hello World code sample [*Click to open or collapse the listing*] +[%collapsible] +==== +.... +include::devguide:example$java/StartUsingCapella.java[] +.... +==== + +Otherwise, read on as we introduce the CRUD API and connection to Capella or self-managed Couchbase Server. + +TIP: There's a *View* link to the complete sample code on GitHub above each of the snippets on these SDK pages, and a *Copy* icon to grab just the snippet shown. + + +== Connect to your Database + +Connect to your Couchbase Capella operational cluster (or your local Couchbase Cluster, if you are trying out self-managed Couchbase). + + +[tabs,sync-group-id="Couchbase Capella|Self-Managed Couchbase Server"] +==== +Couchbase Capella:: ++ +-- +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tags=connect,indent=0] +---- +-- + +Self-Managed Couchbase Server:: ++ +-- + +[source,java] +---- +include::devguide:example$java/StartUsing.java[tags="connect-info",indent=0] +---- +[source,java] +---- +include::devguide:example$java/StartUsing.java[tags="connect-env",indent=0] +---- +-- + +Cloud Native Gateway (CNG):: ++ +-- +Couchbase's large number of ports across the URLs of many services can be proxied by using a `couchbase2://` endpoint as the connection string -- +currently only compatible with recent versions of xref:operator:ROOT:concept-cloud-native-gateway.adoc[Couchbase Autonomous Operator]: + +[source,scala] +---- +Cluster cluster = Cluster.connect( + "couchbase2://10.12.14.16", + ClusterOptions + .create(username, password) + .environment(env) +) +---- + +Read more on the xref:howtos:managing-connections.adoc#cloud-native-gateway[Connections] page. +-- + +Quarkus:: ++ +-- +Our xref:quarkus-extension:ROOT:overview.adoc[Couchbase Quarkus Java Extension docs] cover installing and connecting with the Quarkus extension in detail, +but if you already have Quarkus installed and a project ready (with `quarkus-couchbase` in your `pom.xml` or `build.gradle`), +then your `src/main/resources/application.properties` file needs to contain: + +[source,properties] +---- +quarkus.couchbase.connection-string=localhost +quarkus.couchbase.username=username +quarkus.couchbase.password=password +---- +-- +==== + + +For a deeper look at connection options, read xref:howtos:managing-connections.adoc[]. + +TIP: The connection code for getting started uses the Administrator password that you were given during set up. +In any production app you should create a role restricted to the permissions needed for your app -- +more on this in xref:concept-docs:best-practices.adoc#roles-and-rbac[the Security documentation]. + + +The `ClusterEnvironment.Builder` is covered more fully on the xref:ref:client-settings.adoc#the-environment-builder[Client Settings] page. + +[TIP] +.Simpler Connection +==== +There's also a simpler version of `Cluster.connect()` for when you don't need to customize the cluster environment: +[source,java] +---- +include::devguide:example$java/SimpleConnect.java[tags=connect-string,indent=0] +---- +==== + + +=== Opening a Bucket + +Following successful authentication, open the bucket with: + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=bucket,indent=0] +---- + + + +//// +client has not yet completed bootstrapping (connecting to the cluster, getting the configuration, connecting to the nodes). +//// + + +`waitUntilReady` is an optional call, +but it is good practice to use it. +Opening resources such as buckets is asynchronous -- +that is, the `cluster.bucket` call returns immediately and proceeds in the background. +`waitUntilReady` ensures that the bucket resource is fully loaded before proceeding. +If not present, then the first key-value (KV) operation on the bucket will wait for it to be ready. +As with the earlier `Cluster.connect`, we use `.get` on the result here for simplicity. + +*Collections* allow documents to be grouped by purpose or theme, according to a specified *scope* -- see data modeling, <>. +Here we will use the `airport` collection within the `inventory` scope from `travel-sample` bucket as an example. + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=collection,indent=0] +---- + + +== Create, Read, Update, Delete + +Couchbase documents are organized into buckets, scopes, and collections. +https://en.wikipedia.org/wiki/CRUD[CRUD operations^] -- Create, Read, Update, Delete -- can be performed upon documents in a collection. + +=== JSON + +We'll create a snippet of JSON to work with, using the client's own JSON library, +but you can read about the Scala SDK's support for other JSON libraries on the +xref:howtos:json.adoc[] page. + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=json,indent=0] +---- + +=== Insert (Create) and Upsert + +`insert` and `upsert` will both create a new document. +The difference between the two is that if a document with that key already exists, the `insert` operation will fail, +// throwing `DocumentExistsException` -- +while the `upsert` operation will succeed, replacing the content. + +We need to provide a unique ID as the key, and we'll use a UUID here: + +.Creating a new document +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=upsert,indent=0] +---- + +=== Get (Read) + +The `get` method reads a document from a collection. +// If the collection does not have a document with this ID, the `get` method also throws `DocumentNotFoundException`. + +Wrapping the method in a `Try` / `Catch` is a good way to handle exceptions: + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=get,indent=0] +---- + +//// +Here we're fetching the value for the key `docId`, +converting that value to a `JsonObjectSafe` +(a simple wrapper around `JsonObject` that returns `Try` results -- see +xref:howtos:json.adoc#error-handling-and-jsonobjectsafe[JsonObjectSafe] for details), +and then accessing the value of the *status* key as a String. +//// + +//// +==== Better Error Handling + +All three of these operations could fail, so there's quite a lot of error handling code here to do something quite simple. +One way to improve on this is by using `flatMap`, like this: + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=get-map,indent=0] +---- + +Alternatively, you can use a for-comprehension, like so: + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=get-for,indent=0] +---- + +Either of these methods will stop on the first failed operation. So the final returned `Try` contains either +a) `Success` and the result of the final operation, indicating that everything was successful, or +b) `Failure` with the error returned by the first failing operation. +//// + +=== Replace (Update) and Overloads + +//// +You'll notice that most operations in the Scala SDK have two overloads. +One will take an Options builder, which provides all possible options that operation takes. +For instance: +//// + +.The replace method updates the value of an existing document +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=replace-options,indent=0] +---- + +//// +These options blocks are implemented as Scala case classes: +they are immutable data objects that return a copy of themselves on each change. + +The other overload is provided purely for convenience. +It takes named arguments instead of an Options object, and provides only the most commonly used options: + +[source,java] +---- +include::devguide:example$java/StartUsingCapella.java[tag=replace-named,indent=0] +---- +//// + +CAUTION: When you replace a document, it's usually good practice to use xref:howtos:kv-operations.adoc#optimistic-locking[optimistic locking]. +Otherwise, changes might get lost if two people change the same document at the same time. + +=== Remove (Delete) + +The remove method deletes a document from a collection: + +[source,java] +---- +try { + collection.remove("my-document"); +} catch (DocumentNotFoundException ex) { + System.out.println("Document did not exist when trying to remove"); +} +---- + +Like `replace`, `remove` also optionally takes the CAS value if you want to make sure you are only removing the document if it hasn’t changed since you last fetched it. + + +== Data Modeling + +Documents are organized into collections -- collections of documents that belong together. +You get to decide what it means to "belong." +Developers usually put documents of the same type in the same collection. + +For example, imagine you have two types of documents: customers and invoices. +You could put the customer documents in a collection called `customers`, and the invoice documents in a collection called `invoices`. + +Each document belongs to exactly one collection. +A document's ID is unique _within_ the collection. + +Different scopes can hold collections with different names. +There is no relationship between collections in different scopes. +Each collection belongs to just one scope and +a collection's name is unique within the scope. + + +More details can be found on the xref:concept-docs:data-model.adoc[Data Model page]. +// The xref:student-record-developer-tutorial.adoc[Student Records Tutorial] gives an introduction to data modeling in a document database alongside using the {name-sdk}. + + +== What Next? + +//// +Query and KV - +both +- but different emphasis on modelling objects.... +//// + + + +=== Help and Troubleshooting + +* xref:howtos:troubleshooting-cloud-connections.adoc[Troubleshooting common network problems]. +* {forum-link}[Help forum]. +* https://discord.com/channels/915294689681362954/1217642561645318194[Discord channel]. +* Read the xref:howtos:error-handling.adoc[error handling page]. +* xref:cloud:get-started:capella-iq/get-started-with-iq.adoc#generate-sdk-code-preview[Get help from Couchbase iQ]. + +=== Next Steps + +// * Take a look at the xref:hello-world:sample-application.adoc[]. +* xref:concept-docs:data-durability-acid-transactions.adoc[Learn more about the Data Service]. +* xref:concept-docs:querying-your-data.adoc[Discover SQL++] -- our SQL-family querying language. +* Explore some of the xref:project-docs:third-party-integrations.adoc[third party integrations] with Couchbase and the {name_platform} SDK, across the {name_platform} ecosystem. + + + +== Next Steps + +Now you're up and running, try one of the following: + +* Our xref:hello-world:sample-application.adoc[Travel Sample Application] demonstrates all the basics you need to know; +* Explore xref:howtos:kv-operations.adoc[Key Value Operations] (CRUD) against a document database; +* Or xref:howtos:sqlpp-queries-with-sdk.adoc[Query] with our SQL-based {sqlpp} query language; +* Try longer-running queries with our xref:howtos:analytics-using-sdk.adoc[Analytics Service]; +* A xref:howtos:full-text-searching-with-sdk.adoc[Full Text Search]; +* Or read up on xref:concept-docs:data-services.adoc[which service fits your use case]. + +=== Additional Resources + +//// +The Scala SDK includes three APIs. +The examples above show the simple blocking API, for simplicity, but you can also perform all operations in an async style using Scala `Future`, and a reactive style using Project Reactor `SMono` and `SFlux`. +Please see xref:howtos:concurrent-async-apis.adoc[Choosing an API] for more details. +//// + +The API reference is generated for each release and the latest can be found https://docs.couchbase.com/sdk-api/couchbase-scala-client/com/couchbase/client/scala/index.html[here]. + +Couchbase welcomes community contributions to the Java SDK. +The SDK source code is available on https://github.com/couchbase/couchbase-jvm-clients[GitHub]. + +=== Troubleshooting + +* Couchbase Server is designed to work in the same WAN or availability zone as the client application. +If you're running the SDK on your laptop against a Capella cluster, see further information on: +** Notes on xref:ref:client-settings.adoc#constrained-network-environments[Constrained Network Environments]. +** xref:project-docs:compatibility.adoc#network-requirements[Network Requirements]. +** If you have a consumer-grade router which has problems with DNS-SRV records review our xref:howtos:troubleshooting-cloud-connections.adoc#troubleshooting-host-not-found[Troubleshooting Guide]. +* Our https://www.couchbase.com/forums/c/java-sdk/5[community forum] is a great source of help. +//// + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + += Start Using the Java SDK +:description: A quick start guide to get you up and running with Couchbase and the Java SDK. +:page-partial: +:page-aliases: ROOT:getting-started,ROOT:start-using,ROOT:hello-couchbase,ROOT:start-using-sdk,ROOT:java-intro,ROOT:tutorial +:page-toclevels: 2 +:page-pagination: next +:page-topic-type: tutorial + +include::project-docs:partial$attributes.adoc[] + +[abstract] +{description} + +The Couchbase Java client allows applications to access a Couchbase cluster. +It offers synchronous APIs as well as reactive and asynchronous equivalents to maximize flexibility and performance. + +In this guide, you will learn: + +* How to xref:connect[connect to Couchbase Capella or Couchbase Server]. + +* How to xref:add-and-retrieve-documents[add and retrieve Documents]. + +* How to xref:sql-lookup[lookup documents] with the {sqlpp_url}[{sqlpp} (formerly N1QL)] query language. + +== Hello Couchbase + +We will go through the code sample step by step, but for those in a hurry, here's the complete code: + +[{tabs}] +==== +Couchbase Capella Sample:: ++ +-- +If you are connecting to https://docs.couchbase.com/cloud/index.html[Couchbase Capella], you'll need to know the endpoint address, as well as a username and password. + +This example requires the Travel Sample Bucket. +The Couchbase Capella free tier version comes with this bucket, and its Query indexes, loaded and ready. + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tag=cloud-connect,indent=0] +---- +-- + +Local Couchbase Server:: ++ +-- +Before running this example, you will need to install the Travel Sample Bucket +using either the xref:{version-server}@server:manage:manage-settings/install-sample-buckets.adoc#install-sample-buckets-with-the-ui[Web interface] +or the xref:{version-server}@server:manage:manage-settings/install-sample-buckets.adoc#install-sample-buckets-with-the-cli[command line]. + +[source,java] +---- +include:devguide::example$StartUsing.java[tags=start-using,indent=0] +---- +-- +==== + +== Quick Installation + +We recommend running the latest Java LTS version (i.e. at the time of writing JDK {java_latest_lts_version}) with the highest patch version available. +// Other supported Java versions will work, too. +Couchbase publishes all stable artifacts to https://central.sonatype.com/namespace/com.couchbase.client[Maven Central]. + +The latest version (as of June 2024) is https://central.sonatype.com/artifact/com.couchbase.client/java-client/{sdk_current_version}/jar[{sdk_current_version}]. + +You can use your favorite dependency management tool to install the SDK. + +[{tabs}] +==== +Maven:: ++ +-- +[source,xml,subs="+attributes"] +---- + + + com.couchbase.client + java-client + {sdk_current_version} + + +---- +-- + +Gradle:: ++ +-- +[source,groovy,subs="+attributes"] +---- +implementation 'com.couchbase.client:java-client:{sdk_current_version}' +---- +-- +==== + +See the xref:project-docs:sdk-full-installation.adoc[installation page] for more detailed instructions. + + +== Prerequisites + +The following code samples assume: + +[{tabs}] +==== +Couchbase Capella:: ++ +-- +* You have signed up to https://cloud.couchbase.com/sign-up[Couchbase Capella]. + +* You have created your own bucket, or loaded the Travel Sample dataset. +Note, the Travel Sample dataset is installed automatically when deploying a Capella free tier cluster. + +* A user is created with permissions to access the cluster (at least Application Access permissions). +See the xref:cloud:get-started:cluster-and-data.adoc#credentials[Capella connection page] for more details. + +IMPORTANT: Couchbase Capella uses xref:cloud:organizations:organization-projects-overview.adoc[Roles] to control user access to cluster resources. +For the purposes of this guide, you can use the *Organization Owner* role automatically assigned to your account during installation of the Capella cluster. +In a production scenario, we strongly recommend setting up users with more granular access roles as a best practice. +-- + +Local Couchbase Server:: ++ +-- +* xref:{version-server}@server:getting-started/do-a-quick-install.adoc[Couchbase Server] is installed and accessible locally. + +* You have created your own bucket, or loaded the Travel Sample dataset using the xref:{version-server}@server:manage:manage-settings/install-sample-buckets.adoc#install-sample-buckets-with-the-ui[Web interface]. + +* A user is created with permissions to access your cluster (at least Application Access permissions). +See xref:{version-server}@server:manage:manage-security/manage-users-and-roles.adoc[Manage Users, Groups and Roles] for more details. + +IMPORTANT: Couchbase Server uses xref:{version-server}@server:learn:security/roles.adoc[Role Based Access Control (RBAC)] to control access to resources. +In this guide we suggest using the *Full Admin* role created during setup of your local Couchbase Server cluster. +For production client code, you will want to use more appropriate, restrictive settings. +-- +==== + +== Step by Step + +Here are all the imports needed to run the sample code: + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tags=imports,indent=0] +---- + +If you haven't already, create an empty class and add a `main()` method. + +[source, java] +---- +public class YourClassName { + public static void main(String... args) {} +} +---- + +Make sure to replace `YourClassName` with your own class name. + +Above the `main()` method, add the following variables and update them accordingly: + +[{tabs}] +==== +Couchbase Capella:: ++ +-- +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tags=connect-info,indent=0] +---- +-- + +Local Couchbase Server:: ++ +-- + +[source,java] +---- +include::devguide:example$StartUsing.java[tags=connect-info,indent=0] +---- +-- +==== + +In the following sections we will populate the `main()` method. + +=== Connect + +Connect to your cluster by calling the `Cluster.connect()` method and pass it your connection details. +The basic connection details that you’ll need are given below -- for more background information, see xref:howtos:managing-connections.adoc[]. + +[{tabs}] +==== +Couchbase Capella:: ++ +-- +From version 3.3, the Java SDK includes Capella’s standard Certificate Authority (CA) certificates by default, so you don't need any additional configuration. +Capella requires TLS, which you can enable by using a connection string that starts with `couchbases://` (note the final 's'). + +This example shows how to connect and customize the xref:howtos:managing-connections.adoc#cluster-environment[Cluster Environment] settings. + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tags=connect-env,indent=0] +---- + +When accessing Capella from a different Wide Area Network or Availability Zone, you may experience latency issues with the default connection settings. +SDK 3.4 introduces a `wan-development` Configuration Profile, which provides pre-configured timeout settings suitable for working in high latency environments. +Basic usage is shown in the example above, but if you want to learn more see xref:ref:client-settings.adoc#constrained-network-environments[Constrained Network Environments]. + +CAUTION: The Configuration Profiles feature is currently a xref:java-sdk:project-docs:compatibility.adoc#interface-stability[Volatile API] and may be subject to change. +-- + +Local Couchbase Server:: ++ +-- +For developing locally on the same machine as Couchbase Server, your connection string can be `couchbase://127.0.0.1` as shown here. +For production deployments, you will want to enable TLS by using `couchbases://` (note the final 's') instead of `couchbase://`. + +This example shows how to connect and customize the xref:howtos:managing-connections.adoc#cluster-environment[Cluster Environment] settings. + +[source,java] +---- +include::devguide:example$StartUsing.java[tags=connect-env,indent=0] +---- +-- +==== + +[TIP] +.Simpler Connection +==== +There's also a simpler version of `Cluster.connect()` for when you don't need to customize the cluster environment: + +[source,java] +---- +include::devguide:example$SimpleConnect.java[tags=connect-string,indent=0] +---- +==== + +Now that you have a `Cluster`, add this code snippet to access your `Bucket`: + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tag=bucket,indent=0] +---- + + + +//// +[TIP] +.Connecting to Cloud Native Gateway, for Kubernetes or OpenShift +==== +Couchbase's large number of ports across the URLs of many services can be proxied by using a `couchbase2://` endpoint as the connection string -- read more on the xref:howtos:managing-connections.adoc#cloud-native-gateway[Connections] page. +==== +//// + + + + +=== Add and Retrieve Documents + +The Java SDK supports full integration with the xref:{version-server}@server:learn:data/scopes-and-collections.adoc[Collections] feature introduced in Couchbase Server 7.0. +*Collections* allow documents to be grouped by purpose or theme, according to a specified *Scope*. + +Here we refer to the `users` collection within the `tenant_agent_00` scope from the Travel Sample bucket as an example, but you may replace this with your own data. + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tag=collection,indent=0] +---- + +xref:howtos:kv-operations.adoc[Data operations], like storing and retrieving documents, can be done using simple methods on the `Collection` class such as `Collection.get()` and `Collection.upsert()`. + +Add the following code to create a new document and retrieve it: + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tag=upsert-get,indent=0] +---- + +=== {sqlpp} Lookup + +Couchbase {sqlpp} queries can be performed at the `Cluster` or `Scope` level by invoking `Cluster.query()` or `Scope.query()`. + +Cluster level queries require you to specify the fully qualified keyspace each time (e.g. `travel-sample.inventory.airline`). +However, with a Scope level query you only need to specify the Collection name -- which in this case is `airline`: + +[source,java] +---- +include::devguide:example$StartUsingCapella.java[tag=n1ql-query,indent=0] +---- + +You can learn more about {sqlpp} queries on the xref:howtos:n1ql-queries-with-sdk.adoc[] page. + +=== Execute! + +Now that you've completed all the steps, run the example via your IDE or through the command line. +You should expect to see the following output: + +[source, console] +---- +mike +[{"airline":{"country":"United States","iata":"Q5","name":"40-Mile Air","callsign":"MILE-AIR","icao":"MLA","id":10,"type":"airline"}}] +---- + +== Next Steps + +Now you're up and running, try one of the following: + +* Our xref:hello-world:sample-application.adoc[Travel Sample Application] demonstrates all the basics you need to know; +* Explore xref:howtos:kv-operations.adoc[] against a document database; +* Or xref:howtos:n1ql-queries-with-sdk.adoc[] with our {sqlpp} query language; +// * Try longer-running queries with our xref:howtos:analytics-using-sdk.adoc[Analytics Service]; +// * A xref:howtos:full-text-searching-with-sdk.adoc[Full Text Search]; +* Or read up on xref:concept-docs:data-services.adoc[which service fits your use case]. + +=== Additional Resources + +The API reference is generated for each release and the latest can be found http://docs.couchbase.com/sdk-api/couchbase-java-client/[here]. +Older API references are linked from their respective sections in the xref:project-docs:sdk-release-notes.adoc[Release Notes]. + +Couchbase welcomes community contributions to the Java SDK. +The Java SDK source code is available on https://github.com/couchbaselabs/couchbase-jvm-clients[GitHub]. + +If you are planning to use Spring Data Couchbase, see the xref:project-docs:compatibility.adoc#spring-compat[notes on version compatibility]. + +=== Troubleshooting + +* Couchbase Server is designed to work in the same WAN or availability zone as the client application. +If you're running the SDK on your laptop against a Capella cluster, see further information on: +** Notes on xref:ref:client-settings.adoc#constrained-network-environments[Constrained Network Environments]. +** xref:project-docs:compatibility.adoc#network-requirements[Network Requirements]. +** If you have a consumer-grade router which has problems with DNS-SRV records review our xref:howtos:troubleshooting-cloud-connections.adoc#troubleshooting-host-not-found[Troubleshooting Guide]. +* Our https://forums.couchbase.com/c/java-sdk/5[community forum] is a great source of help. +//// + + + + + + + + + + +//// = Start Using the Python SDK :page-aliases: ROOT:getting-started,ROOT:start-using,ROOT:hello-couchbase,ROOT:start-using-sdk :description: Get up and running quickly, installing the Couchbase Python SDK, and running our Hello World example. @@ -421,3 +1329,4 @@ If you're running the SDK on your laptop against a Capella cluster, see further ** xref:project-docs:compatibility.adoc#network-requirements[Network Requirements]. ** If you have a consumer-grade router which has problems with DNS-SRV records review our xref:howtos:troubleshooting-cloud-connections.adoc#troubleshooting-host-not-found[Troubleshooting Guide]. * Our https://forums.couchbase.com/c/python-sdk/10[community forum] is a great source of help. +////