Skip to content

Commit 56bba24

Browse files
DOCSP-11807 doc for http store (#60)
* DOCSP-11807 doc for http store * DOCSP-11807 updates for copy review feedback * DOCSP-11807 updates for tech review feedback * DOCSP-11807 updates for tech review feedback * DOCSP-11807 updates for PM feedback * DOCSP-11807 minor fix
1 parent 25a905a commit 56bba24

15 files changed

+674
-179
lines changed

source/admin/query-with-sql.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Querying with SQL
66

77
.. default-domain:: mongodb
88

9-
.. include:: /includes/fact-sql-beta-message.rst
9+
.. include:: /includes/extracts/fact-sql-beta-message.rst
1010

1111
{+adl+} supports SQL format queries through the :ref:`JDBC driver
1212
<jdbc-driver>` for {+adl+} and using the :ref:`adl-sql-stage`
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
ref: fact-sql-beta-message
2+
content: |
3+
4+
.. note:: Beta
5+
6+
The support for SQL format queries is available as a Beta feature.
7+
The feature and the corresponding documentation may change at any
8+
time during the Beta stage.
9+
---
10+
ref: fact-http-beta-message
11+
content: |
12+
13+
.. note:: Beta
14+
15+
The support for HTTP data stores is available as a Beta feature.
16+
The feature and the corresponding documentation may change at any
17+
time during the Beta stage.
18+
...
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
ref: param-default-format
2+
content: |
3+
4+
.. datalakeconf:: databases.[n].collections.[n].dataSources.[n].defaultFormat
5+
6+
*Optional.* Specifies the default format {+data-lake-short+} assumes
7+
if it encounters a file without an extension while searching the
8+
:datalakeconf:`~databases.[n].collections.[n].dataSources.[n].storeName`.
9+
10+
If omitted, {+data-lake-short+} attempts to detect the file type by
11+
processing a few bytes of the file.
12+
13+
.. note::
14+
15+
If your file format is ``CSV`` or ``TSV``, you must include a header
16+
row in your data. See :ref:`data-lake-csv-tsv-data` for more
17+
information.
18+
19+
The following values are valid for the ``defaultFormat`` field:
20+
21+
``.json, .json.gz, .bson, .bson.gz, .avro, .avro.gz, .orc, .tsv, .tsv.gz,
22+
.csv, .csv.gz, .parquet``
23+
24+
.. seealso::
25+
26+
:ref:`data-lake-data-formats`
27+
---
28+
ref: cli-param-collection-name
29+
content: |
30+
Either the name of the collection to which {+data-lake-short+} maps the data
31+
contained in the {+data-lake-store+} or the wildcard ``"*"`` to dynamically
32+
create collections.
33+
---
34+
ref: cli-param-default-format
35+
content: |
36+
The format that {+data-lake-short+} defaults to if it encounters a file
37+
without an extension while querying the {+data-lake-store+}. If omitted,
38+
{+data-lake-short+} attempts to detect the file type by processing a few
39+
bytes of the file. The following values are valid:
40+
41+
``.json, .json.gz, .bson, .bson.gz, .avro, .avro.gz, .orc, .tsv, .tsv.gz,
42+
.csv, .csv.gz, .parquet``
43+
...

source/includes/fact-sql-beta-message.rst

Lines changed: 0 additions & 6 deletions
This file was deleted.

source/query/query-data-lake.txt

Lines changed: 63 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -138,11 +138,70 @@ define:
138138

139139
To learn more about these settings, see :ref:`datalake-configuration-file`.
140140

141-
{+dl+} creates the virtual databases and collections you specified in your
142-
{+dl+} configuration for the data in your |service| cluster. When you
141+
{+dl+} automatically detects the file format and creates the virtual databases
142+
and collections you specified in your {+dl+} configuration. When you
143143
:doc:`connect </tutorial/connect>` to your {+dl+} and run queries, {+dl+}
144-
processes your queries against the against the data and returns the query
145-
results.
144+
processes your queries against the data and returns the query results.
145+
146+
.. _query-http:
147+
148+
Querying Data at a |http| |url|
149+
-------------------------------
150+
151+
.. include:: /includes/extracts/fact-http-beta-message.rst
152+
153+
You can use {+adl+} to query and analyze data in files hosted at publicly
154+
accessible |url|\s using MongoDB Query Language (MQL). To learn more about the
155+
supported data formats, see :ref:`data-lake-data-formats`. {+adl+} supports
156+
most, but not all the standard server commands. To learn more about the
157+
supported and unsupported MongoDB server commands and aggregation pipleline
158+
stages, see :ref:`data-lake-mql-support`.
159+
160+
To query data in your publicly accessible |url|\s, your {+dl+} storage
161+
:ref:`configuration <datalake-configuration-file>` must contain settings that
162+
define:
163+
164+
- Your |http| {+data-lake-store+}.
165+
- {+dl+} virtual databases and collections that map to your {+data-lake-store+}.
166+
167+
.. example::
168+
169+
.. code-block:: json
170+
171+
{
172+
"stores" : [
173+
{
174+
"name" : "<store-name>",
175+
"provider": "http",
176+
"urls": ["<url>"]
177+
}
178+
],
179+
"databases" : [
180+
{
181+
"name" : "<database-name>",
182+
"collections" : [
183+
{
184+
"name" : "<collection-name>",
185+
"dataSources" : [
186+
{
187+
"storeName" : "<store-name>",
188+
"defaultFormat" : "<file-format>",
189+
"urls" : ["<url>"]
190+
}
191+
]
192+
}
193+
]
194+
}
195+
]
196+
}
197+
198+
To learn more about these settings, see :ref:`datalake-configuration-file`.
199+
200+
{+dl+} creates the virtual databases and collections you specified in your
201+
{+dl+} configuration for the data in your |url|. {+dl+} also creates one
202+
partition for each |url| in your collection. When you :doc:`connect
203+
</tutorial/connect>` to your {+dl+} and run queries, {+dl+} processes your
204+
queries against the data and returns the query results.
146205

147206
.. _federated-queries:
148207

0 commit comments

Comments
 (0)