initial commit

This commit is contained in:
2025-09-01 22:12:29 +02:00
parent b1873f9c1d
commit 02a54f61c0
5598 changed files with 903558 additions and 0 deletions

202
server/node_modules/@google-cloud/storage/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

295
server/node_modules/@google-cloud/storage/README.md generated vendored Normal file
View File

@@ -0,0 +1,295 @@
[//]: # "This README.md file is auto-generated, all changes to this file will be lost."
[//]: # "To regenerate it, use `python -m synthtool`."
<img src="https://avatars2.githubusercontent.com/u/2810941?v=3&s=96" alt="Google Cloud Platform logo" title="Google Cloud Platform" align="right" height="96" width="96"/>
# [Google Cloud Storage: Node.js Client](https://github.com/googleapis/nodejs-storage)
[![release level](https://img.shields.io/badge/release%20level-stable-brightgreen.svg?style=flat)](https://cloud.google.com/terms/launch-stages)
[![npm version](https://img.shields.io/npm/v/@google-cloud/storage.svg)](https://www.npmjs.org/package/@google-cloud/storage)
> Node.js idiomatic client for [Cloud Storage][product-docs].
[Cloud Storage](https://cloud.google.com/storage/docs) allows world-wide
storage and retrieval of any amount of data at any time. You can use Google
Cloud Storage for a range of scenarios including serving website content,
storing data for archival and disaster recovery, or distributing large data
objects to users via direct download.
A comprehensive list of changes in each version may be found in
[the CHANGELOG](https://github.com/googleapis/nodejs-storage/blob/main/CHANGELOG.md).
* [Google Cloud Storage Node.js Client API Reference][client-docs]
* [Google Cloud Storage Documentation][product-docs]
* [github.com/googleapis/nodejs-storage](https://github.com/googleapis/nodejs-storage)
Read more about the client libraries for Cloud APIs, including the older
Google APIs Client Libraries, in [Client Libraries Explained][explained].
[explained]: https://cloud.google.com/apis/docs/client-libraries-explained
**Table of contents:**
* [Quickstart](#quickstart)
* [Before you begin](#before-you-begin)
* [Installing the client library](#installing-the-client-library)
* [Using the client library](#using-the-client-library)
* [Samples](#samples)
* [Versioning](#versioning)
* [Contributing](#contributing)
* [License](#license)
## Quickstart
### Before you begin
1. [Select or create a Cloud Platform project][projects].
1. [Enable billing for your project][billing].
1. [Enable the Google Cloud Storage API][enable_api].
1. [Set up authentication][auth] so you can access the
API from your local workstation.
### Installing the client library
```bash
npm install @google-cloud/storage
```
### Using the client library
```javascript
// Imports the Google Cloud client library
const {Storage} = require('@google-cloud/storage');
// For more information on ways to initialize Storage, please see
// https://googleapis.dev/nodejs/storage/latest/Storage.html
// Creates a client using Application Default Credentials
const storage = new Storage();
// Creates a client from a Google service account key
// const storage = new Storage({keyFilename: 'key.json'});
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// The ID of your GCS bucket
// const bucketName = 'your-unique-bucket-name';
async function createBucket() {
// Creates the new bucket
await storage.createBucket(bucketName);
console.log(`Bucket ${bucketName} created.`);
}
createBucket().catch(console.error);
```
## Samples
Samples are in the [`samples/`](https://github.com/googleapis/nodejs-storage/tree/main/samples) directory. Each sample's `README.md` has instructions for running its sample.
| Sample | Source Code | Try it |
| --------------------------- | --------------------------------- | ------ |
| Add Bucket Conditional Binding | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addBucketConditionalBinding.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addBucketConditionalBinding.js,samples/README.md) |
| Add Bucket Default Owner Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addBucketDefaultOwnerAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addBucketDefaultOwnerAcl.js,samples/README.md) |
| Add Bucket Iam Member | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addBucketIamMember.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addBucketIamMember.js,samples/README.md) |
| Storage Add Bucket Label. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addBucketLabel.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addBucketLabel.js,samples/README.md) |
| Add Bucket Owner Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addBucketOwnerAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addBucketOwnerAcl.js,samples/README.md) |
| Bucket Website Configuration. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addBucketWebsiteConfiguration.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addBucketWebsiteConfiguration.js,samples/README.md) |
| Add File Owner Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/addFileOwnerAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/addFileOwnerAcl.js,samples/README.md) |
| Storage Get Bucket Metadata. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/bucketMetadata.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/bucketMetadata.js,samples/README.md) |
| Change Bucket's Default Storage Class. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/changeDefaultStorageClass.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/changeDefaultStorageClass.js,samples/README.md) |
| Storage File Convert CSEK to CMEK. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/changeFileCSEKToCMEK.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/changeFileCSEKToCMEK.js,samples/README.md) |
| Storage Combine files. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/composeFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/composeFile.js,samples/README.md) |
| Storage Configure Bucket Cors. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/configureBucketCors.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/configureBucketCors.js,samples/README.md) |
| Configure Retries | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/configureRetries.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/configureRetries.js,samples/README.md) |
| Copy File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/copyFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/copyFile.js,samples/README.md) |
| Copy Old Version Of File. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/copyOldVersionOfFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/copyOldVersionOfFile.js,samples/README.md) |
| Create a Dual-Region Bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createBucketWithDualRegion.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createBucketWithDualRegion.js,samples/README.md) |
| Create a hierarchical namespace enabled bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createBucketWithHierarchicalNamespace.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createBucketWithHierarchicalNamespace.js,samples/README.md) |
| Create a Bucket with object retention enabled. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createBucketWithObjectRetention.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createBucketWithObjectRetention.js,samples/README.md) |
| Create Bucket With Storage Class and Location. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createBucketWithStorageClassAndLocation.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createBucketWithStorageClassAndLocation.js,samples/README.md) |
| Create Bucket With Turbo Replication | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createBucketWithTurboReplication.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createBucketWithTurboReplication.js,samples/README.md) |
| Create New Bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createNewBucket.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createNewBucket.js,samples/README.md) |
| Create Notification | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/createNotification.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/createNotification.js,samples/README.md) |
| Delete Bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/deleteBucket.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/deleteBucket.js,samples/README.md) |
| Delete File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/deleteFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/deleteFile.js,samples/README.md) |
| Delete Notification | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/deleteNotification.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/deleteNotification.js,samples/README.md) |
| Delete Old Version Of File. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/deleteOldVersionOfFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/deleteOldVersionOfFile.js,samples/README.md) |
| Disable Bucket Lifecycle Management | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/disableBucketLifecycleManagement.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/disableBucketLifecycleManagement.js,samples/README.md) |
| Storage Disable Bucket Versioning. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/disableBucketVersioning.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/disableBucketVersioning.js,samples/README.md) |
| Disable Default Event Based Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/disableDefaultEventBasedHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/disableDefaultEventBasedHold.js,samples/README.md) |
| Disable Requester Pays | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/disableRequesterPays.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/disableRequesterPays.js,samples/README.md) |
| Disable Uniform Bucket Level Access | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/disableUniformBucketLevelAccess.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/disableUniformBucketLevelAccess.js,samples/README.md) |
| Download Byte Range | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadByteRange.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadByteRange.js,samples/README.md) |
| Download Encrypted File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadEncryptedFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadEncryptedFile.js,samples/README.md) |
| Download File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadFile.js,samples/README.md) |
| Download a File in Chunks With Transfer Manager | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadFileInChunksWithTransferManager.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadFileInChunksWithTransferManager.js,samples/README.md) |
| Download File Using Requester Pays | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadFileUsingRequesterPays.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadFileUsingRequesterPays.js,samples/README.md) |
| Download Folder With Transfer Manager | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadFolderWithTransferManager.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadFolderWithTransferManager.js,samples/README.md) |
| Download Into Memory | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadIntoMemory.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadIntoMemory.js,samples/README.md) |
| Download Many Files With Transfer Manager | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadManyFilesWithTransferManager.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadManyFilesWithTransferManager.js,samples/README.md) |
| Storage Download Public File. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/downloadPublicFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/downloadPublicFile.js,samples/README.md) |
| Enable Bucket Lifecycle Management | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/enableBucketLifecycleManagement.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/enableBucketLifecycleManagement.js,samples/README.md) |
| Storage Enable Bucket Versioning. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/enableBucketVersioning.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/enableBucketVersioning.js,samples/README.md) |
| Enable Default Event Based Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/enableDefaultEventBasedHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/enableDefaultEventBasedHold.js,samples/README.md) |
| Enable Default KMS Key | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/enableDefaultKMSKey.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/enableDefaultKMSKey.js,samples/README.md) |
| Enable Requester Pays | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/enableRequesterPays.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/enableRequesterPays.js,samples/README.md) |
| Enable Uniform Bucket Level Access | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/enableUniformBucketLevelAccess.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/enableUniformBucketLevelAccess.js,samples/README.md) |
| Change File's Storage Class. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/fileChangeStorageClass.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/fileChangeStorageClass.js,samples/README.md) |
| Storage Set File Metadata. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/fileSetMetadata.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/fileSetMetadata.js,samples/README.md) |
| Generate Encryption Key | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/generateEncryptionKey.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/generateEncryptionKey.js,samples/README.md) |
| Generate Signed Url | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/generateSignedUrl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/generateSignedUrl.js,samples/README.md) |
| Generate V4 Read Signed Url | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/generateV4ReadSignedUrl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/generateV4ReadSignedUrl.js,samples/README.md) |
| Generate V4 Signed Policy | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/generateV4SignedPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/generateV4SignedPolicy.js,samples/README.md) |
| Generate V4 Upload Signed Url | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/generateV4UploadSignedUrl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/generateV4UploadSignedUrl.js,samples/README.md) |
| Get Autoclass | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getAutoclass.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getAutoclass.js,samples/README.md) |
| Get Default Event Based Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getDefaultEventBasedHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getDefaultEventBasedHold.js,samples/README.md) |
| Get Metadata | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getMetadata.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getMetadata.js,samples/README.md) |
| Get Metadata Notifications | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getMetadataNotifications.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getMetadataNotifications.js,samples/README.md) |
| Get Public Access Prevention | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getPublicAccessPrevention.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getPublicAccessPrevention.js,samples/README.md) |
| Get RPO | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getRPO.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getRPO.js,samples/README.md) |
| Get Requester Pays Status | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getRequesterPaysStatus.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getRequesterPaysStatus.js,samples/README.md) |
| Get Retention Policy | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getRetentionPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getRetentionPolicy.js,samples/README.md) |
| Storage Get Service Account. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getServiceAccount.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getServiceAccount.js,samples/README.md) |
| Get Soft Deleted Bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getSoftDeletedBucket.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getSoftDeletedBucket.js,samples/README.md) |
| Get Uniform Bucket Level Access | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/getUniformBucketLevelAccess.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/getUniformBucketLevelAccess.js,samples/README.md) |
| Activate HMAC SA Key. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/hmacKeyActivate.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/hmacKeyActivate.js,samples/README.md) |
| Create HMAC SA Key. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/hmacKeyCreate.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/hmacKeyCreate.js,samples/README.md) |
| Deactivate HMAC SA Key. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/hmacKeyDeactivate.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/hmacKeyDeactivate.js,samples/README.md) |
| Delete HMAC SA Key. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/hmacKeyDelete.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/hmacKeyDelete.js,samples/README.md) |
| Get HMAC SA Key Metadata. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/hmacKeyGet.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/hmacKeyGet.js,samples/README.md) |
| List HMAC SA Keys Metadata. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/hmacKeysList.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/hmacKeysList.js,samples/README.md) |
| List Buckets | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listBuckets.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listBuckets.js,samples/README.md) |
| List Files | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listFiles.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listFiles.js,samples/README.md) |
| List Files By Prefix | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listFilesByPrefix.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listFilesByPrefix.js,samples/README.md) |
| List Files Paginate | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listFilesPaginate.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listFilesPaginate.js,samples/README.md) |
| List Files with Old Versions. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listFilesWithOldVersions.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listFilesWithOldVersions.js,samples/README.md) |
| List Notifications | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listNotifications.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listNotifications.js,samples/README.md) |
| List Soft Deleted Bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/listSoftDeletedBucket.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/listSoftDeletedBucket.js,samples/README.md) |
| Lock Retention Policy | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/lockRetentionPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/lockRetentionPolicy.js,samples/README.md) |
| Storage Make Bucket Public. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/makeBucketPublic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/makeBucketPublic.js,samples/README.md) |
| Make Public | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/makePublic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/makePublic.js,samples/README.md) |
| Move File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/moveFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/moveFile.js,samples/README.md) |
| Move File Atomic | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/moveFileAtomic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/moveFileAtomic.js,samples/README.md) |
| Print Bucket Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printBucketAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printBucketAcl.js,samples/README.md) |
| Print Bucket Acl For User | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printBucketAclForUser.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printBucketAclForUser.js,samples/README.md) |
| Print File Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printFileAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printFileAcl.js,samples/README.md) |
| Print File Acl For User | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/printFileAclForUser.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/printFileAclForUser.js,samples/README.md) |
| Quickstart | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/quickstart.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/quickstart.js,samples/README.md) |
| Release Event Based Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/releaseEventBasedHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/releaseEventBasedHold.js,samples/README.md) |
| Release Temporary Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/releaseTemporaryHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/releaseTemporaryHold.js,samples/README.md) |
| Remove Bucket Conditional Binding | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeBucketConditionalBinding.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeBucketConditionalBinding.js,samples/README.md) |
| Storage Remove Bucket Cors Configuration. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeBucketCors.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeBucketCors.js,samples/README.md) |
| Remove Bucket Default Owner | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeBucketDefaultOwner.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeBucketDefaultOwner.js,samples/README.md) |
| Remove Bucket Iam Member | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeBucketIamMember.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeBucketIamMember.js,samples/README.md) |
| Storage Remove Bucket Label. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeBucketLabel.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeBucketLabel.js,samples/README.md) |
| Remove Bucket Owner Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeBucketOwnerAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeBucketOwnerAcl.js,samples/README.md) |
| Remove Default KMS Key. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeDefaultKMSKey.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeDefaultKMSKey.js,samples/README.md) |
| Remove File Owner Acl | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeFileOwnerAcl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeFileOwnerAcl.js,samples/README.md) |
| Remove Retention Policy | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/removeRetentionPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/removeRetentionPolicy.js,samples/README.md) |
| Rename File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/renameFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/renameFile.js,samples/README.md) |
| Restore Soft Deleted Bucket | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/restoreSoftDeletedBucket.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/restoreSoftDeletedBucket.js,samples/README.md) |
| Rotate Encryption Key | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/rotateEncryptionKey.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/rotateEncryptionKey.js,samples/README.md) |
| Set Autoclass | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setAutoclass.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setAutoclass.js,samples/README.md) |
| Set Client Endpoint | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setClientEndpoint.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setClientEndpoint.js,samples/README.md) |
| Set Event Based Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setEventBasedHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setEventBasedHold.js,samples/README.md) |
| Set the object retention policy of a File. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setObjectRetentionPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setObjectRetentionPolicy.js,samples/README.md) |
| Set Public Access Prevention Enforced | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setPublicAccessPreventionEnforced.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setPublicAccessPreventionEnforced.js,samples/README.md) |
| Set Public Access Prevention Inherited | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setPublicAccessPreventionInherited.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setPublicAccessPreventionInherited.js,samples/README.md) |
| Set RPO Async Turbo | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setRPOAsyncTurbo.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setRPOAsyncTurbo.js,samples/README.md) |
| Set RPO Default | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setRPODefault.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setRPODefault.js,samples/README.md) |
| Set Retention Policy | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setRetentionPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setRetentionPolicy.js,samples/README.md) |
| Set Temporary Hold | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/setTemporaryHold.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/setTemporaryHold.js,samples/README.md) |
| Stream File Download | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/streamFileDownload.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/streamFileDownload.js,samples/README.md) |
| Stream File Upload | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/streamFileUpload.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/streamFileUpload.js,samples/README.md) |
| Upload a directory to a bucket. | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadDirectory.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadDirectory.js,samples/README.md) |
| Upload Directory With Transfer Manager | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadDirectoryWithTransferManager.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadDirectoryWithTransferManager.js,samples/README.md) |
| Upload Encrypted File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadEncryptedFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadEncryptedFile.js,samples/README.md) |
| Upload File | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadFile.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadFile.js,samples/README.md) |
| Upload a File in Chunks With Transfer Manager | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadFileInChunksWithTransferManager.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadFileInChunksWithTransferManager.js,samples/README.md) |
| Upload File With Kms Key | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadFileWithKmsKey.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadFileWithKmsKey.js,samples/README.md) |
| Upload From Memory | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadFromMemory.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadFromMemory.js,samples/README.md) |
| Upload Many Files With Transfer Manager | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadManyFilesWithTransferManager.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadManyFilesWithTransferManager.js,samples/README.md) |
| Upload Without Authentication | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadWithoutAuthentication.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadWithoutAuthentication.js,samples/README.md) |
| Upload Without Authentication Signed Url | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/uploadWithoutAuthenticationSignedUrl.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/uploadWithoutAuthenticationSignedUrl.js,samples/README.md) |
| View Bucket Iam Members | [source code](https://github.com/googleapis/nodejs-storage/blob/main/samples/viewBucketIamMembers.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-storage&page=editor&open_in_editor=samples/viewBucketIamMembers.js,samples/README.md) |
The [Google Cloud Storage Node.js Client API Reference][client-docs] documentation
also contains samples.
## Supported Node.js Versions
Our client libraries follow the [Node.js release schedule](https://github.com/nodejs/release#release-schedule).
Libraries are compatible with all current _active_ and _maintenance_ versions of
Node.js.
If you are using an end-of-life version of Node.js, we recommend that you update
as soon as possible to an actively supported LTS version.
Google's client libraries support legacy versions of Node.js runtimes on a
best-efforts basis with the following warnings:
* Legacy versions are not tested in continuous integration.
* Some security patches and features cannot be backported.
* Dependencies cannot be kept up-to-date.
Client libraries targeting some end-of-life versions of Node.js are available, and
can be installed through npm [dist-tags](https://docs.npmjs.com/cli/dist-tag).
The dist-tags follow the naming convention `legacy-(version)`.
For example, `npm install @google-cloud/storage@legacy-8` installs client libraries
for versions compatible with Node.js 8.
## Versioning
This library follows [Semantic Versioning](http://semver.org/).
This library is considered to be **stable**. The code surface will not change in backwards-incompatible ways
unless absolutely necessary (e.g. because of critical security issues) or with
an extensive deprecation period. Issues and requests against **stable** libraries
are addressed with the highest priority.
More Information: [Google Cloud Platform Launch Stages][launch_stages]
[launch_stages]: https://cloud.google.com/terms/launch-stages
## Contributing
Contributions welcome! See the [Contributing Guide](https://github.com/googleapis/nodejs-storage/blob/main/CONTRIBUTING.md).
Please note that this `README.md`, the `samples/README.md`,
and a variety of configuration files in this repository (including `.nycrc` and `tsconfig.json`)
are generated from a central template. To edit one of these files, make an edit
to its templates in
[directory](https://github.com/googleapis/synthtool).
## License
Apache Version 2.0
See [LICENSE](https://github.com/googleapis/nodejs-storage/blob/main/LICENSE)
[client-docs]: https://cloud.google.com/nodejs/docs/reference/storage/latest
[product-docs]: https://cloud.google.com/storage
[shell_img]: https://gstatic.com/cloudssh/images/open-btn.png
[projects]: https://console.cloud.google.com/project
[billing]: https://support.google.com/cloud/answer/6293499#enable-billing
[enable_api]: https://console.cloud.google.com/flows/enableapi?apiid=storage-api.googleapis.com
[auth]: https://cloud.google.com/docs/authentication/external/set-up-adc-local

View File

@@ -0,0 +1,3 @@
{
"type": "commonjs"
}

View File

@@ -0,0 +1,165 @@
import { BodyResponseCallback, DecorateRequestOptions, BaseMetadata } from './nodejs-common/index.js';
export interface AclOptions {
pathPrefix: string;
request: (reqOpts: DecorateRequestOptions, callback: BodyResponseCallback) => void;
}
export type GetAclResponse = [
AccessControlObject | AccessControlObject[],
AclMetadata
];
export interface GetAclCallback {
(err: Error | null, acl?: AccessControlObject | AccessControlObject[] | null, apiResponse?: AclMetadata): void;
}
export interface GetAclOptions {
entity: string;
generation?: number;
userProject?: string;
}
export interface UpdateAclOptions {
entity: string;
role: string;
generation?: number;
userProject?: string;
}
export type UpdateAclResponse = [AccessControlObject, AclMetadata];
export interface UpdateAclCallback {
(err: Error | null, acl?: AccessControlObject | null, apiResponse?: AclMetadata): void;
}
export interface AddAclOptions {
entity: string;
role: string;
generation?: number;
userProject?: string;
}
export type AddAclResponse = [AccessControlObject, AclMetadata];
export interface AddAclCallback {
(err: Error | null, acl?: AccessControlObject | null, apiResponse?: AclMetadata): void;
}
export type RemoveAclResponse = [AclMetadata];
export interface RemoveAclCallback {
(err: Error | null, apiResponse?: AclMetadata): void;
}
export interface RemoveAclOptions {
entity: string;
generation?: number;
userProject?: string;
}
export interface AccessControlObject {
entity: string;
role: string;
projectTeam: string;
}
export interface AclMetadata extends BaseMetadata {
bucket?: string;
domain?: string;
entity?: string;
entityId?: string;
generation?: string;
object?: string;
projectTeam?: {
projectNumber?: string;
team?: 'editors' | 'owners' | 'viewers';
};
role?: 'OWNER' | 'READER' | 'WRITER' | 'FULL_CONTROL';
[key: string]: unknown;
}
/**
* Attach functionality to a {@link Storage.acl} instance. This will add an
* object for each role group (owners, readers, and writers), with each object
* containing methods to add or delete a type of entity.
*
* As an example, here are a few methods that are created.
*
* myBucket.acl.readers.deleteGroup('groupId', function(err) {});
*
* myBucket.acl.owners.addUser('email@example.com', function(err, acl) {});
*
* myBucket.acl.writers.addDomain('example.com', function(err, acl) {});
*
* @private
*/
declare class AclRoleAccessorMethods {
private static accessMethods;
private static entities;
private static roles;
owners: {};
readers: {};
writers: {};
constructor();
_assignAccessMethods(role: string): void;
}
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against an
* object or bucket (for example, `READ` or `WRITE`); the entity defines who the
* permission applies to (for example, a specific user or group of users).
*
* Where an `entity` value is accepted, we follow the format the Cloud Storage
* API expects.
*
* Refer to
* https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls
* for the most up-to-date values.
*
* - `user-userId`
* - `user-email`
* - `group-groupId`
* - `group-email`
* - `domain-domain`
* - `project-team-projectId`
* - `allUsers`
* - `allAuthenticatedUsers`
*
* Examples:
*
* - The user "liz@example.com" would be `user-liz@example.com`.
* - The group "example@googlegroups.com" would be
* `group-example@googlegroups.com`.
* - To refer to all members of the Google Apps for Business domain
* "example.com", the entity would be `domain-example.com`.
*
* For more detailed information, see
* {@link http://goo.gl/6qBBPO| About Access Control Lists}.
*
* @constructor Acl
* @mixin
* @param {object} options Configuration options.
*/
declare class Acl extends AclRoleAccessorMethods {
default: Acl;
pathPrefix: string;
request_: (reqOpts: DecorateRequestOptions, callback: BodyResponseCallback) => void;
constructor(options: AclOptions);
add(options: AddAclOptions): Promise<AddAclResponse>;
add(options: AddAclOptions, callback: AddAclCallback): void;
delete(options: RemoveAclOptions): Promise<RemoveAclResponse>;
delete(options: RemoveAclOptions, callback: RemoveAclCallback): void;
get(options?: GetAclOptions): Promise<GetAclResponse>;
get(options: GetAclOptions, callback: GetAclCallback): void;
get(callback: GetAclCallback): void;
update(options: UpdateAclOptions): Promise<UpdateAclResponse>;
update(options: UpdateAclOptions, callback: UpdateAclCallback): void;
/**
* Transform API responses to a consistent object format.
*
* @private
*/
makeAclObject_(accessControlObject: AccessControlObject): AccessControlObject;
/**
* Patch requests up to the bucket's request object.
*
* @private
*
* @param {string} method Action.
* @param {string} path Request path.
* @param {*} query Request query object.
* @param {*} body Request body contents.
* @param {function} callback Callback function.
*/
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
}
export { Acl, AclRoleAccessorMethods };

View File

@@ -0,0 +1,718 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Object.defineProperty(exports, "__esModule", { value: true });
exports.AclRoleAccessorMethods = exports.Acl = void 0;
const promisify_1 = require("@google-cloud/promisify");
/**
* Attach functionality to a {@link Storage.acl} instance. This will add an
* object for each role group (owners, readers, and writers), with each object
* containing methods to add or delete a type of entity.
*
* As an example, here are a few methods that are created.
*
* myBucket.acl.readers.deleteGroup('groupId', function(err) {});
*
* myBucket.acl.owners.addUser('email@example.com', function(err, acl) {});
*
* myBucket.acl.writers.addDomain('example.com', function(err, acl) {});
*
* @private
*/
class AclRoleAccessorMethods {
constructor() {
this.owners = {};
this.readers = {};
this.writers = {};
/**
* An object of convenience methods to add or delete owner ACL permissions
* for a given entity.
*
* The supported methods include:
*
* - `myFile.acl.owners.addAllAuthenticatedUsers`
* - `myFile.acl.owners.deleteAllAuthenticatedUsers`
* - `myFile.acl.owners.addAllUsers`
* - `myFile.acl.owners.deleteAllUsers`
* - `myFile.acl.owners.addDomain`
* - `myFile.acl.owners.deleteDomain`
* - `myFile.acl.owners.addGroup`
* - `myFile.acl.owners.deleteGroup`
* - `myFile.acl.owners.addProject`
* - `myFile.acl.owners.deleteProject`
* - `myFile.acl.owners.addUser`
* - `myFile.acl.owners.deleteUser`
*
* @name Acl#owners
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* //-
* // Add a user as an owner of a file.
* //-
* const myBucket = gcs.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
* myFile.acl.owners.addUser('email@example.com', function(err, aclObject)
* {});
*
* //-
* // For reference, the above command is the same as running the following.
* //-
* myFile.acl.add({
* entity: 'user-email@example.com',
* role: gcs.acl.OWNER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.owners.addUser('email@example.com').then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
this.owners = {};
/**
* An object of convenience methods to add or delete reader ACL permissions
* for a given entity.
*
* The supported methods include:
*
* - `myFile.acl.readers.addAllAuthenticatedUsers`
* - `myFile.acl.readers.deleteAllAuthenticatedUsers`
* - `myFile.acl.readers.addAllUsers`
* - `myFile.acl.readers.deleteAllUsers`
* - `myFile.acl.readers.addDomain`
* - `myFile.acl.readers.deleteDomain`
* - `myFile.acl.readers.addGroup`
* - `myFile.acl.readers.deleteGroup`
* - `myFile.acl.readers.addProject`
* - `myFile.acl.readers.deleteProject`
* - `myFile.acl.readers.addUser`
* - `myFile.acl.readers.deleteUser`
*
* @name Acl#readers
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* //-
* // Add a user as a reader of a file.
* //-
* myFile.acl.readers.addUser('email@example.com', function(err, aclObject)
* {});
*
* //-
* // For reference, the above command is the same as running the following.
* //-
* myFile.acl.add({
* entity: 'user-email@example.com',
* role: gcs.acl.READER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.readers.addUser('email@example.com').then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
this.readers = {};
/**
* An object of convenience methods to add or delete writer ACL permissions
* for a given entity.
*
* The supported methods include:
*
* - `myFile.acl.writers.addAllAuthenticatedUsers`
* - `myFile.acl.writers.deleteAllAuthenticatedUsers`
* - `myFile.acl.writers.addAllUsers`
* - `myFile.acl.writers.deleteAllUsers`
* - `myFile.acl.writers.addDomain`
* - `myFile.acl.writers.deleteDomain`
* - `myFile.acl.writers.addGroup`
* - `myFile.acl.writers.deleteGroup`
* - `myFile.acl.writers.addProject`
* - `myFile.acl.writers.deleteProject`
* - `myFile.acl.writers.addUser`
* - `myFile.acl.writers.deleteUser`
*
* @name Acl#writers
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* //-
* // Add a user as a writer of a file.
* //-
* myFile.acl.writers.addUser('email@example.com', function(err, aclObject)
* {});
*
* //-
* // For reference, the above command is the same as running the following.
* //-
* myFile.acl.add({
* entity: 'user-email@example.com',
* role: gcs.acl.WRITER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.writers.addUser('email@example.com').then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
this.writers = {};
AclRoleAccessorMethods.roles.forEach(this._assignAccessMethods.bind(this));
}
_assignAccessMethods(role) {
const accessMethods = AclRoleAccessorMethods.accessMethods;
const entities = AclRoleAccessorMethods.entities;
const roleGroup = role.toLowerCase() + 's';
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this[roleGroup] = entities.reduce((acc, entity) => {
const isPrefix = entity.charAt(entity.length - 1) === '-';
accessMethods.forEach(accessMethod => {
let method = accessMethod + entity[0].toUpperCase() + entity.substring(1);
if (isPrefix) {
method = method.replace('-', '');
}
// Wrap the parent accessor method (e.g. `add` or `delete`) to avoid the
// more complex API of specifying an `entity` and `role`.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
acc[method] = (entityId, options, callback) => {
let apiEntity;
if (typeof options === 'function') {
callback = options;
options = {};
}
if (isPrefix) {
apiEntity = entity + entityId;
}
else {
// If the entity is not a prefix, it is a special entity group
// that does not require further details. The accessor methods
// only accept a callback.
apiEntity = entity;
callback = entityId;
}
options = Object.assign({
entity: apiEntity,
role,
}, options);
const args = [options];
if (typeof callback === 'function') {
args.push(callback);
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return this[accessMethod].apply(this, args);
};
});
return acc;
}, {});
}
}
exports.AclRoleAccessorMethods = AclRoleAccessorMethods;
AclRoleAccessorMethods.accessMethods = ['add', 'delete'];
AclRoleAccessorMethods.entities = [
// Special entity groups that do not require further specification.
'allAuthenticatedUsers',
'allUsers',
// Entity groups that require specification, e.g. `user-email@example.com`.
'domain-',
'group-',
'project-',
'user-',
];
AclRoleAccessorMethods.roles = ['OWNER', 'READER', 'WRITER'];
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against an
* object or bucket (for example, `READ` or `WRITE`); the entity defines who the
* permission applies to (for example, a specific user or group of users).
*
* Where an `entity` value is accepted, we follow the format the Cloud Storage
* API expects.
*
* Refer to
* https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls
* for the most up-to-date values.
*
* - `user-userId`
* - `user-email`
* - `group-groupId`
* - `group-email`
* - `domain-domain`
* - `project-team-projectId`
* - `allUsers`
* - `allAuthenticatedUsers`
*
* Examples:
*
* - The user "liz@example.com" would be `user-liz@example.com`.
* - The group "example@googlegroups.com" would be
* `group-example@googlegroups.com`.
* - To refer to all members of the Google Apps for Business domain
* "example.com", the entity would be `domain-example.com`.
*
* For more detailed information, see
* {@link http://goo.gl/6qBBPO| About Access Control Lists}.
*
* @constructor Acl
* @mixin
* @param {object} options Configuration options.
*/
class Acl extends AclRoleAccessorMethods {
constructor(options) {
super();
this.pathPrefix = options.pathPrefix;
this.request_ = options.request;
}
/**
* @typedef {array} AddAclResponse
* @property {object} 0 The Acl Objects.
* @property {object} 1 The full API response.
*/
/**
* @callback AddAclCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The Acl Objects.
* @param {object} apiResponse The full API response.
*/
/**
* Add access controls on a {@link Bucket} or {@link File}.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/insert| BucketAccessControls: insert API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/insert| ObjectAccessControls: insert API Documentation}
*
* @param {object} options Configuration options.
* @param {string} options.entity Whose permissions will be added.
* @param {string} options.role Permissions allowed for the defined entity.
* See {@link https://cloud.google.com/storage/docs/access-control Access
* Control}.
* @param {number} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {AddAclCallback} [callback] Callback function.
* @returns {Promise<AddAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* const options = {
* entity: 'user-useremail@example.com',
* role: gcs.acl.OWNER_ROLE
* };
*
* myBucket.acl.add(options, function(err, aclObject, apiResponse) {});
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* // Here is how you would grant ownership permissions to a user on a
* specific
* // revision of a file.
* //-
* myFile.acl.add({
* entity: 'user-useremail@example.com',
* role: gcs.acl.OWNER_ROLE,
* generation: 1
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myBucket.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_file_owner
* Example of adding an owner to a file:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_owner
* Example of adding an owner to a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_default_owner
* Example of adding a default owner to a bucket:
*/
add(options, callback) {
const query = {};
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
this.request({
method: 'POST',
uri: '',
qs: query,
maxRetries: 0, //explicitly set this value since this is a non-idempotent function
json: {
entity: options.entity,
role: options.role.toUpperCase(),
},
}, (err, resp) => {
if (err) {
callback(err, null, resp);
return;
}
callback(null, this.makeAclObject_(resp), resp);
});
}
/**
* @typedef {array} RemoveAclResponse
* @property {object} 0 The full API response.
*/
/**
* @callback RemoveAclCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
/**
* Delete access controls on a {@link Bucket} or {@link File}.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/delete| BucketAccessControls: delete API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/delete| ObjectAccessControls: delete API Documentation}
*
* @param {object} options Configuration object.
* @param {string} options.entity Whose permissions will be revoked.
* @param {int} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {RemoveAclCallback} callback The callback function.
* @returns {Promise<RemoveAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* myBucket.acl.delete({
* entity: 'user-useremail@example.com'
* }, function(err, apiResponse) {});
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* //-
* myFile.acl.delete({
* entity: 'user-useremail@example.com',
* generation: 1
* }, function(err, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.delete().then(function(data) {
* const apiResponse = data[0];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_owner
* Example of removing an owner from a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_default_owner
* Example of removing a default owner from a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_file_owner
* Example of removing an owner from a bucket:
*/
delete(options, callback) {
const query = {};
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
this.request({
method: 'DELETE',
uri: '/' + encodeURIComponent(options.entity),
qs: query,
}, (err, resp) => {
callback(err, resp);
});
}
/**
* @typedef {array} GetAclResponse
* @property {object|object[]} 0 Single or array of Acl Objects.
* @property {object} 1 The full API response.
*/
/**
* @callback GetAclCallback
* @param {?Error} err Request error, if any.
* @param {object|object[]} acl Single or array of Acl Objects.
* @param {object} apiResponse The full API response.
*/
/**
* Get access controls on a {@link Bucket} or {@link File}. If
* an entity is omitted, you will receive an array of all applicable access
* controls.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/get| BucketAccessControls: get API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/get| ObjectAccessControls: get API Documentation}
*
* @param {object|function} [options] Configuration options. If you want to
* receive a list of all access controls, pass the callback function as
* the only argument.
* @param {string} options.entity Whose permissions will be fetched.
* @param {number} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {GetAclCallback} [callback] Callback function.
* @returns {Promise<GetAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* myBucket.acl.get({
* entity: 'user-useremail@example.com'
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // Get all access controls.
* //-
* myBucket.acl.get(function(err, aclObjects, apiResponse) {
* // aclObjects = [
* // {
* // entity: 'user-useremail@example.com',
* // role: 'owner'
* // }
* // ]
* });
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* //-
* myFile.acl.get({
* entity: 'user-useremail@example.com',
* generation: 1
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myBucket.acl.get().then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_file_acl
* Example of printing a file's ACL:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_file_acl_for_user
* Example of printing a file's ACL for a specific user:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl
* Example of printing a bucket's ACL:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl_for_user
* Example of printing a bucket's ACL for a specific user:
*/
get(optionsOrCallback, cb) {
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : null;
const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : cb;
let path = '';
const query = {};
if (options) {
path = '/' + encodeURIComponent(options.entity);
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
}
this.request({
uri: path,
qs: query,
}, (err, resp) => {
if (err) {
callback(err, null, resp);
return;
}
let results;
if (resp.items) {
results = resp.items.map(this.makeAclObject_);
}
else {
results = this.makeAclObject_(resp);
}
callback(null, results, resp);
});
}
/**
* @typedef {array} UpdateAclResponse
* @property {object} 0 The updated Acl Objects.
* @property {object} 1 The full API response.
*/
/**
* @callback UpdateAclCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The updated Acl Objects.
* @param {object} apiResponse The full API response.
*/
/**
* Update access controls on a {@link Bucket} or {@link File}.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/update| BucketAccessControls: update API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/update| ObjectAccessControls: update API Documentation}
*
* @param {object} options Configuration options.
* @param {string} options.entity Whose permissions will be updated.
* @param {string} options.role Permissions allowed for the defined entity.
* See {@link Storage.acl}.
* @param {number} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {UpdateAclCallback} [callback] Callback function.
* @returns {Promise<UpdateAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* const options = {
* entity: 'user-useremail@example.com',
* role: gcs.acl.WRITER_ROLE
* };
*
* myBucket.acl.update(options, function(err, aclObject, apiResponse) {});
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* //-
* myFile.acl.update({
* entity: 'user-useremail@example.com',
* role: gcs.acl.WRITER_ROLE,
* generation: 1
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.update(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
update(options, callback) {
const query = {};
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
this.request({
method: 'PUT',
uri: '/' + encodeURIComponent(options.entity),
qs: query,
json: {
role: options.role.toUpperCase(),
},
}, (err, resp) => {
if (err) {
callback(err, null, resp);
return;
}
callback(null, this.makeAclObject_(resp), resp);
});
}
/**
* Transform API responses to a consistent object format.
*
* @private
*/
makeAclObject_(accessControlObject) {
const obj = {
entity: accessControlObject.entity,
role: accessControlObject.role,
};
if (accessControlObject.projectTeam) {
obj.projectTeam = accessControlObject.projectTeam;
}
return obj;
}
/**
* Patch requests up to the bucket's request object.
*
* @private
*
* @param {string} method Action.
* @param {string} path Request path.
* @param {*} query Request query object.
* @param {*} body Request body contents.
* @param {function} callback Callback function.
*/
request(reqOpts, callback) {
reqOpts.uri = this.pathPrefix + reqOpts.uri;
this.request_(reqOpts, callback);
}
}
exports.Acl = Acl;
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
(0, promisify_1.promisifyAll)(Acl, {
exclude: ['request'],
});

View File

@@ -0,0 +1,824 @@
import { ApiError, BodyResponseCallback, DecorateRequestOptions, DeleteCallback, ExistsCallback, GetConfig, MetadataCallback, ServiceObject, SetMetadataResponse } from './nodejs-common/index.js';
import { RequestResponse } from './nodejs-common/service-object.js';
import * as http from 'http';
import { Acl, AclMetadata } from './acl.js';
import { Channel } from './channel.js';
import { File, FileOptions, CreateResumableUploadOptions, CreateWriteStreamOptions } from './file.js';
import { Iam } from './iam.js';
import { Notification } from './notification.js';
import { Storage, Cors, PreconditionOptions, BucketOptions } from './storage.js';
import { GetSignedUrlResponse, GetSignedUrlCallback, SignerGetSignedUrlConfig, URLSigner, Query } from './signer.js';
import { Readable } from 'stream';
import { CRC32CValidatorGenerator } from './crc32c.js';
import { URL } from 'url';
import { BaseMetadata, SetMetadataOptions } from './nodejs-common/service-object.js';
export type GetFilesResponse = [
File[],
(GetFilesOptions | {}) & Partial<Pick<GetFilesOptions, 'pageToken'>>,
unknown
];
export interface GetFilesCallback {
(err: Error | null, files?: File[], nextQuery?: {}, apiResponse?: unknown): void;
}
interface WatchAllOptions {
delimiter?: string;
maxResults?: number;
pageToken?: string;
prefix?: string;
projection?: string;
userProject?: string;
versions?: boolean;
}
export interface AddLifecycleRuleOptions extends PreconditionOptions {
append?: boolean;
}
export interface LifecycleAction {
type: 'Delete' | 'SetStorageClass' | 'AbortIncompleteMultipartUpload';
storageClass?: string;
}
export interface LifecycleCondition {
age?: number;
createdBefore?: Date | string;
customTimeBefore?: Date | string;
daysSinceCustomTime?: number;
daysSinceNoncurrentTime?: number;
isLive?: boolean;
matchesPrefix?: string[];
matchesSuffix?: string[];
matchesStorageClass?: string[];
noncurrentTimeBefore?: Date | string;
numNewerVersions?: number;
}
export interface LifecycleRule {
action: LifecycleAction;
condition: LifecycleCondition;
}
export interface LifecycleCondition {
age?: number;
createdBefore?: Date | string;
customTimeBefore?: Date | string;
daysSinceCustomTime?: number;
daysSinceNoncurrentTime?: number;
isLive?: boolean;
matchesPrefix?: string[];
matchesSuffix?: string[];
matchesStorageClass?: string[];
noncurrentTimeBefore?: Date | string;
numNewerVersions?: number;
}
export interface LifecycleRule {
action: LifecycleAction;
condition: LifecycleCondition;
}
export interface EnableLoggingOptions extends PreconditionOptions {
bucket?: string | Bucket;
prefix: string;
}
export interface GetFilesOptions {
autoPaginate?: boolean;
delimiter?: string;
endOffset?: string;
includeFoldersAsPrefixes?: boolean;
includeTrailingDelimiter?: boolean;
prefix?: string;
matchGlob?: string;
maxApiCalls?: number;
maxResults?: number;
pageToken?: string;
softDeleted?: boolean;
startOffset?: string;
userProject?: string;
versions?: boolean;
fields?: string;
}
export interface CombineOptions extends PreconditionOptions {
kmsKeyName?: string;
userProject?: string;
}
export interface CombineCallback {
(err: Error | null, newFile: File | null, apiResponse: unknown): void;
}
export type CombineResponse = [File, unknown];
export interface CreateChannelConfig extends WatchAllOptions {
address: string;
}
export interface CreateChannelOptions {
userProject?: string;
}
export type CreateChannelResponse = [Channel, unknown];
export interface CreateChannelCallback {
(err: Error | null, channel: Channel | null, apiResponse: unknown): void;
}
export interface CreateNotificationOptions {
customAttributes?: {
[key: string]: string;
};
eventTypes?: string[];
objectNamePrefix?: string;
payloadFormat?: string;
userProject?: string;
}
export interface CreateNotificationCallback {
(err: Error | null, notification: Notification | null, apiResponse: unknown): void;
}
export type CreateNotificationResponse = [Notification, unknown];
export interface DeleteBucketOptions {
ignoreNotFound?: boolean;
userProject?: string;
}
export type DeleteBucketResponse = [unknown];
export interface DeleteBucketCallback extends DeleteCallback {
(err: Error | null, apiResponse: unknown): void;
}
export interface DeleteFilesOptions extends GetFilesOptions, PreconditionOptions {
force?: boolean;
}
export interface DeleteFilesCallback {
(err: Error | Error[] | null, apiResponse?: object): void;
}
export type DeleteLabelsResponse = [unknown];
export type DeleteLabelsCallback = SetLabelsCallback;
export type DeleteLabelsOptions = PreconditionOptions;
export type DisableRequesterPaysOptions = PreconditionOptions;
export type DisableRequesterPaysResponse = [unknown];
export interface DisableRequesterPaysCallback {
(err?: Error | null, apiResponse?: object): void;
}
export type EnableRequesterPaysResponse = [unknown];
export interface EnableRequesterPaysCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type EnableRequesterPaysOptions = PreconditionOptions;
export interface BucketExistsOptions extends GetConfig {
userProject?: string;
}
export type BucketExistsResponse = [boolean];
export type BucketExistsCallback = ExistsCallback;
export interface GetBucketOptions extends GetConfig {
userProject?: string;
}
export type GetBucketResponse = [Bucket, unknown];
export interface GetBucketCallback {
(err: ApiError | null, bucket: Bucket | null, apiResponse: unknown): void;
}
export interface GetLabelsOptions {
userProject?: string;
}
export type GetLabelsResponse = [unknown];
export interface GetLabelsCallback {
(err: Error | null, labels: object | null): void;
}
export interface RestoreOptions {
generation: string;
projection?: 'full' | 'noAcl';
}
export interface BucketMetadata extends BaseMetadata {
acl?: AclMetadata[] | null;
autoclass?: {
enabled?: boolean;
toggleTime?: string;
terminalStorageClass?: string;
terminalStorageClassUpdateTime?: string;
};
billing?: {
requesterPays?: boolean;
};
cors?: Cors[];
customPlacementConfig?: {
dataLocations?: string[];
};
defaultEventBasedHold?: boolean;
defaultObjectAcl?: AclMetadata[];
encryption?: {
defaultKmsKeyName?: string;
} | null;
hierarchicalNamespace?: {
enabled?: boolean;
};
iamConfiguration?: {
publicAccessPrevention?: string;
uniformBucketLevelAccess?: {
enabled?: boolean;
lockedTime?: string;
};
};
labels?: {
[key: string]: string | null;
};
lifecycle?: {
rule?: LifecycleRule[];
} | null;
location?: string;
locationType?: string;
logging?: {
logBucket?: string;
logObjectPrefix?: string;
};
generation?: string;
metageneration?: string;
name?: string;
objectRetention?: {
mode?: string;
};
owner?: {
entity?: string;
entityId?: string;
};
projectNumber?: string | number;
retentionPolicy?: {
effectiveTime?: string;
isLocked?: boolean;
retentionPeriod?: string | number;
} | null;
rpo?: string;
softDeleteTime?: string;
hardDeleteTime?: string;
softDeletePolicy?: {
retentionDurationSeconds?: string | number;
readonly effectiveTime?: string;
};
storageClass?: string;
timeCreated?: string;
updated?: string;
versioning?: {
enabled?: boolean;
};
website?: {
mainPageSuffix?: string;
notFoundPage?: string;
};
}
export type GetBucketMetadataResponse = [BucketMetadata, unknown];
export interface GetBucketMetadataCallback {
(err: ApiError | null, metadata: BucketMetadata | null, apiResponse: unknown): void;
}
export interface GetBucketMetadataOptions {
userProject?: string;
}
export interface GetBucketSignedUrlConfig extends Pick<SignerGetSignedUrlConfig, 'host' | 'signingEndpoint'> {
action: 'list';
version?: 'v2' | 'v4';
cname?: string;
virtualHostedStyle?: boolean;
expires: string | number | Date;
extensionHeaders?: http.OutgoingHttpHeaders;
queryParams?: Query;
}
export declare enum BucketActionToHTTPMethod {
list = "GET"
}
export declare enum AvailableServiceObjectMethods {
setMetadata = 0,
delete = 1
}
export interface GetNotificationsOptions {
userProject?: string;
}
export interface GetNotificationsCallback {
(err: Error | null, notifications: Notification[] | null, apiResponse: unknown): void;
}
export type GetNotificationsResponse = [Notification[], unknown];
export interface MakeBucketPrivateOptions {
includeFiles?: boolean;
force?: boolean;
metadata?: BucketMetadata;
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type MakeBucketPrivateResponse = [File[]];
export interface MakeBucketPrivateCallback {
(err?: Error | null, files?: File[]): void;
}
export interface MakeBucketPublicOptions {
includeFiles?: boolean;
force?: boolean;
}
export interface MakeBucketPublicCallback {
(err?: Error | null, files?: File[]): void;
}
export type MakeBucketPublicResponse = [File[]];
export interface SetBucketMetadataOptions extends PreconditionOptions {
userProject?: string;
}
export type SetBucketMetadataResponse = [BucketMetadata];
export interface SetBucketMetadataCallback {
(err?: Error | null, metadata?: BucketMetadata): void;
}
export interface BucketLockCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type BucketLockResponse = [unknown];
export interface Labels {
[key: string]: string;
}
export interface SetLabelsOptions extends PreconditionOptions {
userProject?: string;
}
export type SetLabelsResponse = [unknown];
export interface SetLabelsCallback {
(err?: Error | null, metadata?: unknown): void;
}
export interface SetBucketStorageClassOptions extends PreconditionOptions {
userProject?: string;
}
export interface SetBucketStorageClassCallback {
(err?: Error | null): void;
}
export type UploadResponse = [File, unknown];
export interface UploadCallback {
(err: Error | null, file?: File | null, apiResponse?: unknown): void;
}
export interface UploadOptions extends CreateResumableUploadOptions, CreateWriteStreamOptions {
destination?: string | File;
encryptionKey?: string | Buffer;
kmsKeyName?: string;
onUploadProgress?: (progressEvent: any) => void;
}
export interface MakeAllFilesPublicPrivateOptions {
force?: boolean;
private?: boolean;
public?: boolean;
userProject?: string;
}
interface MakeAllFilesPublicPrivateCallback {
(err?: Error | Error[] | null, files?: File[]): void;
}
type MakeAllFilesPublicPrivateResponse = [File[]];
export declare enum BucketExceptionMessages {
PROVIDE_SOURCE_FILE = "You must provide at least one source file.",
DESTINATION_FILE_NOT_SPECIFIED = "A destination file must be specified.",
CHANNEL_ID_REQUIRED = "An ID is required to create a channel.",
TOPIC_NAME_REQUIRED = "A valid topic name is required.",
CONFIGURATION_OBJECT_PREFIX_REQUIRED = "A configuration object with a prefix is required.",
SPECIFY_FILE_NAME = "A file name must be specified.",
METAGENERATION_NOT_PROVIDED = "A metageneration must be provided.",
SUPPLY_NOTIFICATION_ID = "You must supply a notification ID."
}
/**
* @callback Crc32cGeneratorToStringCallback
* A method returning the CRC32C as a base64-encoded string.
*
* @returns {string}
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
/**
* @callback Crc32cGeneratorValidateCallback
* A method validating a base64-encoded CRC32C string.
*
* @param {string} [value] base64-encoded CRC32C string to validate
* @returns {boolean}
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
**/
/**
* @callback Crc32cGeneratorUpdateCallback
* A method for passing `Buffer`s for CRC32C generation.
*
* @param {Buffer} [data] data to update CRC32C value with
* @returns {undefined}
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
**/
/**
* @typedef {object} CRC32CValidator
* @property {Crc32cGeneratorToStringCallback}
* @property {Crc32cGeneratorValidateCallback}
* @property {Crc32cGeneratorUpdateCallback}
*/
/**
* A function that generates a CRC32C Validator. Defaults to {@link CRC32C}
*
* @name Bucket#crc32cGenerator
* @type {CRC32CValidator}
*/
/**
* Get and set IAM policies for your bucket.
*
* @name Bucket#iam
* @mixes Iam
*
* See {@link https://cloud.google.com/storage/docs/access-control/iam#short_title_iam_management| Cloud Storage IAM Management}
* See {@link https://cloud.google.com/iam/docs/granting-changing-revoking-access| Granting, Changing, and Revoking Access}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
*
* //-
* // Get the IAM policy for your bucket.
* //-
* bucket.iam.getPolicy(function(err, policy) {
* console.log(policy);
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.getPolicy().then(function(data) {
* const policy = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_view_bucket_iam_members
* Example of retrieving a bucket's IAM policy:
*
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_add_bucket_iam_member
* Example of adding to a bucket's IAM policy:
*
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_remove_bucket_iam_member
* Example of removing from a bucket's IAM policy:
*/
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against
* an object or bucket (for example, `READ` or `WRITE`); the entity defines
* who the permission applies to (for example, a specific user or group of
* users).
*
* The `acl` object on a Bucket instance provides methods to get you a list of
* the ACLs defined on your bucket, as well as set, update, and delete them.
*
* Buckets also have
* {@link https://cloud.google.com/storage/docs/access-control/lists#default| default ACLs}
* for all created files. Default ACLs specify permissions that all new
* objects added to the bucket will inherit by default. You can add, delete,
* get, and update entities and permissions for these as well with
* {@link Bucket#acl.default}.
*
* See {@link http://goo.gl/6qBBPO| About Access Control Lists}
* See {@link https://cloud.google.com/storage/docs/access-control/lists#default| Default ACLs}
*
* @name Bucket#acl
* @mixes Acl
* @property {Acl} default Cloud Storage Buckets have
* {@link https://cloud.google.com/storage/docs/access-control/lists#default| default ACLs}
* for all created files. You can add, delete, get, and update entities and
* permissions for these as well. The method signatures and examples are all
* the same, after only prefixing the method call with `default`.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Make a bucket's contents publicly readable.
* //-
* const myBucket = storage.bucket('my-bucket');
*
* const options = {
* entity: 'allUsers',
* role: storage.acl.READER_ROLE
* };
*
* myBucket.acl.add(options, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myBucket.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl
* Example of printing a bucket's ACL:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl_for_user
* Example of printing a bucket's ACL for a specific user:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_owner
* Example of adding an owner to a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_owner
* Example of removing an owner from a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_default_owner
* Example of adding a default owner to a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_default_owner
* Example of removing a default owner from a bucket:
*/
/**
* The API-formatted resource description of the bucket.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name Bucket#metadata
* @type {object}
*/
/**
* The bucket's name.
* @name Bucket#name
* @type {string}
*/
/**
* Get {@link File} objects for the files currently in the bucket as a
* readable object stream.
*
* @method Bucket#getFilesStream
* @param {GetFilesOptions} [query] Query object for listing files.
* @returns {ReadableStream} A readable stream that emits {@link File} instances.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
*
* bucket.getFilesStream()
* .on('error', console.error)
* .on('data', function(file) {
* // file is a File object.
* })
* .on('end', function() {
* // All files retrieved.
* });
*
* //-
* // If you anticipate many results, you can end a stream early to prevent
* // unnecessary processing and API requests.
* //-
* bucket.getFilesStream()
* .on('data', function(file) {
* this.end();
* });
*
* //-
* // If you're filtering files with a delimiter, you should use
* // {@link Bucket#getFiles} and set `autoPaginate: false` in order to
* // preserve the `apiResponse` argument.
* //-
* const prefixes = [];
*
* function callback(err, files, nextQuery, apiResponse) {
* prefixes = prefixes.concat(apiResponse.prefixes);
*
* if (nextQuery) {
* bucket.getFiles(nextQuery, callback);
* } else {
* // prefixes = The finished array of prefixes.
* }
* }
*
* bucket.getFiles({
* autoPaginate: false,
* delimiter: '/'
* }, callback);
* ```
*/
/**
* Create a Bucket object to interact with a Cloud Storage bucket.
*
* @class
* @hideconstructor
*
* @param {Storage} storage A {@link Storage} instance.
* @param {string} name The name of the bucket.
* @param {object} [options] Configuration object.
* @param {string} [options.userProject] User project.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* ```
*/
declare class Bucket extends ServiceObject<Bucket, BucketMetadata> {
name: string;
/**
* A reference to the {@link Storage} associated with this {@link Bucket}
* instance.
* @name Bucket#storage
* @type {Storage}
*/
storage: Storage;
/**
* A user project to apply to each request from this bucket.
* @name Bucket#userProject
* @type {string}
*/
userProject?: string;
acl: Acl;
iam: Iam;
crc32cGenerator: CRC32CValidatorGenerator;
getFilesStream(query?: GetFilesOptions): Readable;
signer?: URLSigner;
private instanceRetryValue?;
instancePreconditionOpts?: PreconditionOptions;
constructor(storage: Storage, name: string, options?: BucketOptions);
/**
* The bucket's Cloud Storage URI (`gs://`)
*
* @example
* ```ts
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* // `gs://my-bucket`
* const href = bucket.cloudStorageURI.href;
* ```
*/
get cloudStorageURI(): URL;
addLifecycleRule(rule: LifecycleRule | LifecycleRule[], options?: AddLifecycleRuleOptions): Promise<SetBucketMetadataResponse>;
addLifecycleRule(rule: LifecycleRule | LifecycleRule[], options: AddLifecycleRuleOptions, callback: SetBucketMetadataCallback): void;
addLifecycleRule(rule: LifecycleRule | LifecycleRule[], callback: SetBucketMetadataCallback): void;
combine(sources: string[] | File[], destination: string | File, options?: CombineOptions): Promise<CombineResponse>;
combine(sources: string[] | File[], destination: string | File, options: CombineOptions, callback: CombineCallback): void;
combine(sources: string[] | File[], destination: string | File, callback: CombineCallback): void;
createChannel(id: string, config: CreateChannelConfig, options?: CreateChannelOptions): Promise<CreateChannelResponse>;
createChannel(id: string, config: CreateChannelConfig, callback: CreateChannelCallback): void;
createChannel(id: string, config: CreateChannelConfig, options: CreateChannelOptions, callback: CreateChannelCallback): void;
createNotification(topic: string, options?: CreateNotificationOptions): Promise<CreateNotificationResponse>;
createNotification(topic: string, options: CreateNotificationOptions, callback: CreateNotificationCallback): void;
createNotification(topic: string, callback: CreateNotificationCallback): void;
deleteFiles(query?: DeleteFilesOptions): Promise<void>;
deleteFiles(callback: DeleteFilesCallback): void;
deleteFiles(query: DeleteFilesOptions, callback: DeleteFilesCallback): void;
deleteLabels(labels?: string | string[]): Promise<DeleteLabelsResponse>;
deleteLabels(options: DeleteLabelsOptions): Promise<DeleteLabelsResponse>;
deleteLabels(callback: DeleteLabelsCallback): void;
deleteLabels(labels: string | string[], options: DeleteLabelsOptions): Promise<DeleteLabelsResponse>;
deleteLabels(labels: string | string[], callback: DeleteLabelsCallback): void;
deleteLabels(labels: string | string[], options: DeleteLabelsOptions, callback: DeleteLabelsCallback): void;
disableRequesterPays(options?: DisableRequesterPaysOptions): Promise<DisableRequesterPaysResponse>;
disableRequesterPays(callback: DisableRequesterPaysCallback): void;
disableRequesterPays(options: DisableRequesterPaysOptions, callback: DisableRequesterPaysCallback): void;
enableLogging(config: EnableLoggingOptions): Promise<SetBucketMetadataResponse>;
enableLogging(config: EnableLoggingOptions, callback: SetBucketMetadataCallback): void;
enableRequesterPays(options?: EnableRequesterPaysOptions): Promise<EnableRequesterPaysResponse>;
enableRequesterPays(callback: EnableRequesterPaysCallback): void;
enableRequesterPays(options: EnableRequesterPaysOptions, callback: EnableRequesterPaysCallback): void;
/**
* Create a {@link File} object. See {@link File} to see how to handle
* the different use cases you may have.
*
* @param {string} name The name of the file in this bucket.
* @param {FileOptions} [options] Configuration options.
* @param {string|number} [options.generation] Only use a specific revision of
* this file.
* @param {string} [options.encryptionKey] A custom encryption key. See
* {@link https://cloud.google.com/storage/docs/encryption#customer-supplied| Customer-supplied Encryption Keys}.
* @param {string} [options.kmsKeyName] The name of the Cloud KMS key that will
* be used to encrypt the object. Must be in the format:
* `projects/my-project/locations/location/keyRings/my-kr/cryptoKeys/my-key`.
* KMS key ring must use the same location as the bucket.
* @param {string} [options.userProject] The ID of the project which will be
* billed for all requests made from File object.
* @returns {File}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* const file = bucket.file('my-existing-file.png');
* ```
*/
file(name: string, options?: FileOptions): File;
getFiles(query?: GetFilesOptions): Promise<GetFilesResponse>;
getFiles(query: GetFilesOptions, callback: GetFilesCallback): void;
getFiles(callback: GetFilesCallback): void;
getLabels(options?: GetLabelsOptions): Promise<GetLabelsResponse>;
getLabels(callback: GetLabelsCallback): void;
getLabels(options: GetLabelsOptions, callback: GetLabelsCallback): void;
getNotifications(options?: GetNotificationsOptions): Promise<GetNotificationsResponse>;
getNotifications(callback: GetNotificationsCallback): void;
getNotifications(options: GetNotificationsOptions, callback: GetNotificationsCallback): void;
getSignedUrl(cfg: GetBucketSignedUrlConfig): Promise<GetSignedUrlResponse>;
getSignedUrl(cfg: GetBucketSignedUrlConfig, callback: GetSignedUrlCallback): void;
lock(metageneration: number | string): Promise<BucketLockResponse>;
lock(metageneration: number | string, callback: BucketLockCallback): void;
/**
* @typedef {object} RestoreOptions Options for Bucket#restore(). See an
* {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/restore#resource| Object resource}.
* @param {number} [generation] If present, selects a specific revision of this object.
* @param {string} [projection] Specifies the set of properties to return. If used, must be 'full' or 'noAcl'.
*/
/**
* Restores a soft-deleted bucket
* @param {RestoreOptions} options Restore options.
* @returns {Promise<Bucket>}
*/
restore(options: RestoreOptions): Promise<Bucket>;
makePrivate(options?: MakeBucketPrivateOptions): Promise<MakeBucketPrivateResponse>;
makePrivate(callback: MakeBucketPrivateCallback): void;
makePrivate(options: MakeBucketPrivateOptions, callback: MakeBucketPrivateCallback): void;
makePublic(options?: MakeBucketPublicOptions): Promise<MakeBucketPublicResponse>;
makePublic(callback: MakeBucketPublicCallback): void;
makePublic(options: MakeBucketPublicOptions, callback: MakeBucketPublicCallback): void;
/**
* Get a reference to a Cloud Pub/Sub Notification.
*
* @param {string} id ID of notification.
* @returns {Notification}
* @see Notification
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const notification = bucket.notification('1');
* ```
*/
notification(id: string): Notification;
removeRetentionPeriod(options?: SetBucketMetadataOptions): Promise<SetBucketMetadataResponse>;
removeRetentionPeriod(callback: SetBucketMetadataCallback): void;
removeRetentionPeriod(options: SetBucketMetadataOptions, callback: SetBucketMetadataCallback): void;
request(reqOpts: DecorateRequestOptions): Promise<RequestResponse>;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
setLabels(labels: Labels, options?: SetLabelsOptions): Promise<SetLabelsResponse>;
setLabels(labels: Labels, callback: SetLabelsCallback): void;
setLabels(labels: Labels, options: SetLabelsOptions, callback: SetLabelsCallback): void;
setMetadata(metadata: BucketMetadata, options?: SetMetadataOptions): Promise<SetMetadataResponse<BucketMetadata>>;
setMetadata(metadata: BucketMetadata, callback: MetadataCallback<BucketMetadata>): void;
setMetadata(metadata: BucketMetadata, options: SetMetadataOptions, callback: MetadataCallback<BucketMetadata>): void;
setRetentionPeriod(duration: number, options?: SetBucketMetadataOptions): Promise<SetBucketMetadataResponse>;
setRetentionPeriod(duration: number, callback: SetBucketMetadataCallback): void;
setRetentionPeriod(duration: number, options: SetBucketMetadataOptions, callback: SetBucketMetadataCallback): void;
setCorsConfiguration(corsConfiguration: Cors[], options?: SetBucketMetadataOptions): Promise<SetBucketMetadataResponse>;
setCorsConfiguration(corsConfiguration: Cors[], callback: SetBucketMetadataCallback): void;
setCorsConfiguration(corsConfiguration: Cors[], options: SetBucketMetadataOptions, callback: SetBucketMetadataCallback): void;
setStorageClass(storageClass: string, options?: SetBucketStorageClassOptions): Promise<SetBucketMetadataResponse>;
setStorageClass(storageClass: string, callback: SetBucketStorageClassCallback): void;
setStorageClass(storageClass: string, options: SetBucketStorageClassOptions, callback: SetBucketStorageClassCallback): void;
/**
* Set a user project to be billed for all requests made from this Bucket
* object and any files referenced from this Bucket object.
*
* @param {string} userProject The user project.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
*
* bucket.setUserProject('grape-spaceship-123');
* ```
*/
setUserProject(userProject: string): void;
upload(pathString: string, options?: UploadOptions): Promise<UploadResponse>;
upload(pathString: string, options: UploadOptions, callback: UploadCallback): void;
upload(pathString: string, callback: UploadCallback): void;
makeAllFilesPublicPrivate_(options?: MakeAllFilesPublicPrivateOptions): Promise<MakeAllFilesPublicPrivateResponse>;
makeAllFilesPublicPrivate_(callback: MakeAllFilesPublicPrivateCallback): void;
makeAllFilesPublicPrivate_(options: MakeAllFilesPublicPrivateOptions, callback: MakeAllFilesPublicPrivateCallback): void;
getId(): string;
disableAutoRetryConditionallyIdempotent_(coreOpts: any, methodType: AvailableServiceObjectMethods, localPreconditionOptions?: PreconditionOptions): void;
}
/**
* Reference to the {@link Bucket} class.
* @name module:@google-cloud/storage.Bucket
* @see Bucket
*/
export { Bucket };

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,33 @@
import { BaseMetadata, ServiceObject } from './nodejs-common/index.js';
import { Storage } from './storage.js';
export interface StopCallback {
(err: Error | null, apiResponse?: unknown): void;
}
/**
* Create a channel object to interact with a Cloud Storage channel.
*
* See {@link https://cloud.google.com/storage/docs/object-change-notification| Object Change Notification}
*
* @class
*
* @param {string} id The ID of the channel.
* @param {string} resourceId The resource ID of the channel.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* ```
*/
declare class Channel extends ServiceObject<Channel, BaseMetadata> {
constructor(storage: Storage, id: string, resourceId: string);
stop(): Promise<unknown>;
stop(callback: StopCallback): void;
}
/**
* Reference to the {@link Channel} class.
* @name module:@google-cloud/storage.Channel
* @see Channel
*/
export { Channel };

View File

@@ -0,0 +1,104 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Object.defineProperty(exports, "__esModule", { value: true });
exports.Channel = void 0;
const index_js_1 = require("./nodejs-common/index.js");
const promisify_1 = require("@google-cloud/promisify");
/**
* Create a channel object to interact with a Cloud Storage channel.
*
* See {@link https://cloud.google.com/storage/docs/object-change-notification| Object Change Notification}
*
* @class
*
* @param {string} id The ID of the channel.
* @param {string} resourceId The resource ID of the channel.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* ```
*/
class Channel extends index_js_1.ServiceObject {
constructor(storage, id, resourceId) {
const config = {
parent: storage,
baseUrl: '/channels',
// An ID shouldn't be included in the API requests.
// RE:
// https://github.com/GoogleCloudPlatform/google-cloud-node/issues/1145
id: '',
methods: {
// Only need `request`.
},
};
super(config);
this.metadata.id = id;
this.metadata.resourceId = resourceId;
}
/**
* @typedef {array} StopResponse
* @property {object} 0 The full API response.
*/
/**
* @callback StopCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
/**
* Stop this channel.
*
* @param {StopCallback} [callback] Callback function.
* @returns {Promise<StopResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* channel.stop(function(err, apiResponse) {
* if (!err) {
* // Channel stopped successfully.
* }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* channel.stop().then(function(data) {
* const apiResponse = data[0];
* });
* ```
*/
stop(callback) {
callback = callback || index_js_1.util.noop;
this.request({
method: 'POST',
uri: '/stop',
json: this.metadata,
}, (err, apiResponse) => {
callback(err, apiResponse);
});
}
}
exports.Channel = Channel;
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
(0, promisify_1.promisifyAll)(Channel);

View File

@@ -0,0 +1,143 @@
import { PathLike } from 'fs';
/**
* Ported from {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc#L16-L59 github.com/google/crc32c}
*/
declare const CRC32C_EXTENSIONS: readonly [0, 4067132163, 3778769143, 324072436, 3348797215, 904991772, 648144872, 3570033899, 2329499855, 2024987596, 1809983544, 2575936315, 1296289744, 3207089363, 2893594407, 1578318884, 274646895, 3795141740, 4049975192, 51262619, 3619967088, 632279923, 922689671, 3298075524, 2592579488, 1760304291, 2075979607, 2312596564, 1562183871, 2943781820, 3156637768, 1313733451, 549293790, 3537243613, 3246849577, 871202090, 3878099393, 357341890, 102525238, 4101499445, 2858735121, 1477399826, 1264559846, 3107202533, 1845379342, 2677391885, 2361733625, 2125378298, 820201905, 3263744690, 3520608582, 598981189, 4151959214, 85089709, 373468761, 3827903834, 3124367742, 1213305469, 1526817161, 2842354314, 2107672161, 2412447074, 2627466902, 1861252501, 1098587580, 3004210879, 2688576843, 1378610760, 2262928035, 1955203488, 1742404180, 2511436119, 3416409459, 969524848, 714683780, 3639785095, 205050476, 4266873199, 3976438427, 526918040, 1361435347, 2739821008, 2954799652, 1114974503, 2529119692, 1691668175, 2005155131, 2247081528, 3690758684, 697762079, 986182379, 3366744552, 476452099, 3993867776, 4250756596, 255256311, 1640403810, 2477592673, 2164122517, 1922457750, 2791048317, 1412925310, 1197962378, 3037525897, 3944729517, 427051182, 170179418, 4165941337, 746937522, 3740196785, 3451792453, 1070968646, 1905808397, 2213795598, 2426610938, 1657317369, 3053634322, 1147748369, 1463399397, 2773627110, 4215344322, 153784257, 444234805, 3893493558, 1021025245, 3467647198, 3722505002, 797665321, 2197175160, 1889384571, 1674398607, 2443626636, 1164749927, 3070701412, 2757221520, 1446797203, 137323447, 4198817972, 3910406976, 461344835, 3484808360, 1037989803, 781091935, 3705997148, 2460548119, 1623424788, 1939049696, 2180517859, 1429367560, 2807687179, 3020495871, 1180866812, 410100952, 3927582683, 4182430767, 186734380, 3756733383, 763408580, 1053836080, 3434856499, 2722870694, 1344288421, 1131464017, 2971354706, 1708204729, 2545590714, 2229949006, 1988219213, 680717673, 3673779818, 3383336350, 1002577565, 4010310262, 493091189, 238226049, 4233660802, 2987750089, 1082061258, 1395524158, 2705686845, 1972364758, 2279892693, 2494862625, 1725896226, 952904198, 3399985413, 3656866545, 731699698, 4283874585, 222117402, 510512622, 3959836397, 3280807620, 837199303, 582374963, 3504198960, 68661723, 4135334616, 3844915500, 390545967, 1230274059, 3141532936, 2825850620, 1510247935, 2395924756, 2091215383, 1878366691, 2644384480, 3553878443, 565732008, 854102364, 3229815391, 340358836, 3861050807, 4117890627, 119113024, 1493875044, 2875275879, 3090270611, 1247431312, 2660249211, 1828433272, 2141937292, 2378227087, 3811616794, 291187481, 34330861, 4032846830, 615137029, 3603020806, 3314634738, 939183345, 1776939221, 2609017814, 2295496738, 2058945313, 2926798794, 1545135305, 1330124605, 3173225534, 4084100981, 17165430, 307568514, 3762199681, 888469610, 3332340585, 3587147933, 665062302, 2042050490, 2346497209, 2559330125, 1793573966, 3190661285, 1279665062, 1595330642, 2910671697];
declare const CRC32C_EXTENSION_TABLE: Int32Array<ArrayBuffer>;
/** An interface for CRC32C hashing and validation */
interface CRC32CValidator {
/**
* A method returning the CRC32C as a base64-encoded string.
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
toString: () => string;
/**
* A method validating a base64-encoded CRC32C string.
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
*/
validate: (value: string) => boolean;
/**
* A method for passing `Buffer`s for CRC32C generation.
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
*/
update: (data: Buffer) => void;
}
/** A function that generates a CRC32C Validator */
interface CRC32CValidatorGenerator {
/** Should return a new, ready-to-use `CRC32CValidator` */
(): CRC32CValidator;
}
declare const CRC32C_DEFAULT_VALIDATOR_GENERATOR: CRC32CValidatorGenerator;
declare const CRC32C_EXCEPTION_MESSAGES: {
readonly INVALID_INIT_BASE64_RANGE: (l: number) => string;
readonly INVALID_INIT_BUFFER_LENGTH: (l: number) => string;
readonly INVALID_INIT_INTEGER: (l: number) => string;
};
declare class CRC32C implements CRC32CValidator {
#private;
/**
* Constructs a new `CRC32C` object.
*
* Reconstruction is recommended via the `CRC32C.from` static method.
*
* @param initialValue An initial CRC32C value - a signed 32-bit integer.
*/
constructor(initialValue?: number);
/**
* Calculates a CRC32C from a provided buffer.
*
* Implementation inspired from:
* - {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc github.com/google/crc32c}
* - {@link https://github.com/googleapis/python-crc32c/blob/a595e758c08df445a99c3bf132ee8e80a3ec4308/src/google_crc32c/python.py github.com/googleapis/python-crc32c}
* - {@link https://github.com/googleapis/java-storage/pull/1376/files github.com/googleapis/java-storage}
*
* @param data The `Buffer` to generate the CRC32C from
*/
update(data: Buffer): void;
/**
* Validates a provided input to the current CRC32C value.
*
* @param input A Buffer, `CRC32C`-compatible object, base64-encoded data (string), or signed 32-bit integer
*/
validate(input: Buffer | CRC32CValidator | string | number): boolean;
/**
* Returns a `Buffer` representation of the CRC32C value
*/
toBuffer(): Buffer;
/**
* Returns a JSON-compatible, base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify `JSON#stringify`}
*/
toJSON(): string;
/**
* Returns a base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/toString `Object#toString`}
*/
toString(): string;
/**
* Returns the `number` representation of the CRC32C value as a signed 32-bit integer
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/valueOf `Object#valueOf`}
*/
valueOf(): number;
static readonly CRC32C_EXTENSIONS: readonly [0, 4067132163, 3778769143, 324072436, 3348797215, 904991772, 648144872, 3570033899, 2329499855, 2024987596, 1809983544, 2575936315, 1296289744, 3207089363, 2893594407, 1578318884, 274646895, 3795141740, 4049975192, 51262619, 3619967088, 632279923, 922689671, 3298075524, 2592579488, 1760304291, 2075979607, 2312596564, 1562183871, 2943781820, 3156637768, 1313733451, 549293790, 3537243613, 3246849577, 871202090, 3878099393, 357341890, 102525238, 4101499445, 2858735121, 1477399826, 1264559846, 3107202533, 1845379342, 2677391885, 2361733625, 2125378298, 820201905, 3263744690, 3520608582, 598981189, 4151959214, 85089709, 373468761, 3827903834, 3124367742, 1213305469, 1526817161, 2842354314, 2107672161, 2412447074, 2627466902, 1861252501, 1098587580, 3004210879, 2688576843, 1378610760, 2262928035, 1955203488, 1742404180, 2511436119, 3416409459, 969524848, 714683780, 3639785095, 205050476, 4266873199, 3976438427, 526918040, 1361435347, 2739821008, 2954799652, 1114974503, 2529119692, 1691668175, 2005155131, 2247081528, 3690758684, 697762079, 986182379, 3366744552, 476452099, 3993867776, 4250756596, 255256311, 1640403810, 2477592673, 2164122517, 1922457750, 2791048317, 1412925310, 1197962378, 3037525897, 3944729517, 427051182, 170179418, 4165941337, 746937522, 3740196785, 3451792453, 1070968646, 1905808397, 2213795598, 2426610938, 1657317369, 3053634322, 1147748369, 1463399397, 2773627110, 4215344322, 153784257, 444234805, 3893493558, 1021025245, 3467647198, 3722505002, 797665321, 2197175160, 1889384571, 1674398607, 2443626636, 1164749927, 3070701412, 2757221520, 1446797203, 137323447, 4198817972, 3910406976, 461344835, 3484808360, 1037989803, 781091935, 3705997148, 2460548119, 1623424788, 1939049696, 2180517859, 1429367560, 2807687179, 3020495871, 1180866812, 410100952, 3927582683, 4182430767, 186734380, 3756733383, 763408580, 1053836080, 3434856499, 2722870694, 1344288421, 1131464017, 2971354706, 1708204729, 2545590714, 2229949006, 1988219213, 680717673, 3673779818, 3383336350, 1002577565, 4010310262, 493091189, 238226049, 4233660802, 2987750089, 1082061258, 1395524158, 2705686845, 1972364758, 2279892693, 2494862625, 1725896226, 952904198, 3399985413, 3656866545, 731699698, 4283874585, 222117402, 510512622, 3959836397, 3280807620, 837199303, 582374963, 3504198960, 68661723, 4135334616, 3844915500, 390545967, 1230274059, 3141532936, 2825850620, 1510247935, 2395924756, 2091215383, 1878366691, 2644384480, 3553878443, 565732008, 854102364, 3229815391, 340358836, 3861050807, 4117890627, 119113024, 1493875044, 2875275879, 3090270611, 1247431312, 2660249211, 1828433272, 2141937292, 2378227087, 3811616794, 291187481, 34330861, 4032846830, 615137029, 3603020806, 3314634738, 939183345, 1776939221, 2609017814, 2295496738, 2058945313, 2926798794, 1545135305, 1330124605, 3173225534, 4084100981, 17165430, 307568514, 3762199681, 888469610, 3332340585, 3587147933, 665062302, 2042050490, 2346497209, 2559330125, 1793573966, 3190661285, 1279665062, 1595330642, 2910671697];
static readonly CRC32C_EXTENSION_TABLE: Int32Array<ArrayBuffer>;
/**
* Generates a `CRC32C` from a compatible buffer format.
*
* @param value 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`
*/
private static fromBuffer;
static fromFile(file: PathLike): Promise<CRC32C>;
/**
* Generates a `CRC32C` from 4-byte base64-encoded data (string).
*
* @param value 4-byte base64-encoded data (string)
*/
private static fromString;
/**
* Generates a `CRC32C` from a safe, unsigned 32-bit integer.
*
* @param value an unsigned 32-bit integer
*/
private static fromNumber;
/**
* Generates a `CRC32C` from a variety of compatable types.
* Note: strings are treated as input, not as file paths to read from.
*
* @param value A number, 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`, or 4-byte base64-encoded data (string)
*/
static from(value: ArrayBuffer | ArrayBufferView | CRC32CValidator | string | number): CRC32C;
}
export { CRC32C, CRC32C_DEFAULT_VALIDATOR_GENERATOR, CRC32C_EXCEPTION_MESSAGES, CRC32C_EXTENSIONS, CRC32C_EXTENSION_TABLE, CRC32CValidator, CRC32CValidatorGenerator, };

View File

@@ -0,0 +1,261 @@
"use strict";
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _CRC32C_crc32c;
Object.defineProperty(exports, "__esModule", { value: true });
exports.CRC32C_EXTENSION_TABLE = exports.CRC32C_EXTENSIONS = exports.CRC32C_EXCEPTION_MESSAGES = exports.CRC32C_DEFAULT_VALIDATOR_GENERATOR = exports.CRC32C = void 0;
const fs_1 = require("fs");
/**
* Ported from {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc#L16-L59 github.com/google/crc32c}
*/
const CRC32C_EXTENSIONS = [
0x00000000, 0xf26b8303, 0xe13b70f7, 0x1350f3f4, 0xc79a971f, 0x35f1141c,
0x26a1e7e8, 0xd4ca64eb, 0x8ad958cf, 0x78b2dbcc, 0x6be22838, 0x9989ab3b,
0x4d43cfd0, 0xbf284cd3, 0xac78bf27, 0x5e133c24, 0x105ec76f, 0xe235446c,
0xf165b798, 0x030e349b, 0xd7c45070, 0x25afd373, 0x36ff2087, 0xc494a384,
0x9a879fa0, 0x68ec1ca3, 0x7bbcef57, 0x89d76c54, 0x5d1d08bf, 0xaf768bbc,
0xbc267848, 0x4e4dfb4b, 0x20bd8ede, 0xd2d60ddd, 0xc186fe29, 0x33ed7d2a,
0xe72719c1, 0x154c9ac2, 0x061c6936, 0xf477ea35, 0xaa64d611, 0x580f5512,
0x4b5fa6e6, 0xb93425e5, 0x6dfe410e, 0x9f95c20d, 0x8cc531f9, 0x7eaeb2fa,
0x30e349b1, 0xc288cab2, 0xd1d83946, 0x23b3ba45, 0xf779deae, 0x05125dad,
0x1642ae59, 0xe4292d5a, 0xba3a117e, 0x4851927d, 0x5b016189, 0xa96ae28a,
0x7da08661, 0x8fcb0562, 0x9c9bf696, 0x6ef07595, 0x417b1dbc, 0xb3109ebf,
0xa0406d4b, 0x522bee48, 0x86e18aa3, 0x748a09a0, 0x67dafa54, 0x95b17957,
0xcba24573, 0x39c9c670, 0x2a993584, 0xd8f2b687, 0x0c38d26c, 0xfe53516f,
0xed03a29b, 0x1f682198, 0x5125dad3, 0xa34e59d0, 0xb01eaa24, 0x42752927,
0x96bf4dcc, 0x64d4cecf, 0x77843d3b, 0x85efbe38, 0xdbfc821c, 0x2997011f,
0x3ac7f2eb, 0xc8ac71e8, 0x1c661503, 0xee0d9600, 0xfd5d65f4, 0x0f36e6f7,
0x61c69362, 0x93ad1061, 0x80fde395, 0x72966096, 0xa65c047d, 0x5437877e,
0x4767748a, 0xb50cf789, 0xeb1fcbad, 0x197448ae, 0x0a24bb5a, 0xf84f3859,
0x2c855cb2, 0xdeeedfb1, 0xcdbe2c45, 0x3fd5af46, 0x7198540d, 0x83f3d70e,
0x90a324fa, 0x62c8a7f9, 0xb602c312, 0x44694011, 0x5739b3e5, 0xa55230e6,
0xfb410cc2, 0x092a8fc1, 0x1a7a7c35, 0xe811ff36, 0x3cdb9bdd, 0xceb018de,
0xdde0eb2a, 0x2f8b6829, 0x82f63b78, 0x709db87b, 0x63cd4b8f, 0x91a6c88c,
0x456cac67, 0xb7072f64, 0xa457dc90, 0x563c5f93, 0x082f63b7, 0xfa44e0b4,
0xe9141340, 0x1b7f9043, 0xcfb5f4a8, 0x3dde77ab, 0x2e8e845f, 0xdce5075c,
0x92a8fc17, 0x60c37f14, 0x73938ce0, 0x81f80fe3, 0x55326b08, 0xa759e80b,
0xb4091bff, 0x466298fc, 0x1871a4d8, 0xea1a27db, 0xf94ad42f, 0x0b21572c,
0xdfeb33c7, 0x2d80b0c4, 0x3ed04330, 0xccbbc033, 0xa24bb5a6, 0x502036a5,
0x4370c551, 0xb11b4652, 0x65d122b9, 0x97baa1ba, 0x84ea524e, 0x7681d14d,
0x2892ed69, 0xdaf96e6a, 0xc9a99d9e, 0x3bc21e9d, 0xef087a76, 0x1d63f975,
0x0e330a81, 0xfc588982, 0xb21572c9, 0x407ef1ca, 0x532e023e, 0xa145813d,
0x758fe5d6, 0x87e466d5, 0x94b49521, 0x66df1622, 0x38cc2a06, 0xcaa7a905,
0xd9f75af1, 0x2b9cd9f2, 0xff56bd19, 0x0d3d3e1a, 0x1e6dcdee, 0xec064eed,
0xc38d26c4, 0x31e6a5c7, 0x22b65633, 0xd0ddd530, 0x0417b1db, 0xf67c32d8,
0xe52cc12c, 0x1747422f, 0x49547e0b, 0xbb3ffd08, 0xa86f0efc, 0x5a048dff,
0x8ecee914, 0x7ca56a17, 0x6ff599e3, 0x9d9e1ae0, 0xd3d3e1ab, 0x21b862a8,
0x32e8915c, 0xc083125f, 0x144976b4, 0xe622f5b7, 0xf5720643, 0x07198540,
0x590ab964, 0xab613a67, 0xb831c993, 0x4a5a4a90, 0x9e902e7b, 0x6cfbad78,
0x7fab5e8c, 0x8dc0dd8f, 0xe330a81a, 0x115b2b19, 0x020bd8ed, 0xf0605bee,
0x24aa3f05, 0xd6c1bc06, 0xc5914ff2, 0x37faccf1, 0x69e9f0d5, 0x9b8273d6,
0x88d28022, 0x7ab90321, 0xae7367ca, 0x5c18e4c9, 0x4f48173d, 0xbd23943e,
0xf36e6f75, 0x0105ec76, 0x12551f82, 0xe03e9c81, 0x34f4f86a, 0xc69f7b69,
0xd5cf889d, 0x27a40b9e, 0x79b737ba, 0x8bdcb4b9, 0x988c474d, 0x6ae7c44e,
0xbe2da0a5, 0x4c4623a6, 0x5f16d052, 0xad7d5351,
];
exports.CRC32C_EXTENSIONS = CRC32C_EXTENSIONS;
const CRC32C_EXTENSION_TABLE = new Int32Array(CRC32C_EXTENSIONS);
exports.CRC32C_EXTENSION_TABLE = CRC32C_EXTENSION_TABLE;
const CRC32C_DEFAULT_VALIDATOR_GENERATOR = () => new CRC32C();
exports.CRC32C_DEFAULT_VALIDATOR_GENERATOR = CRC32C_DEFAULT_VALIDATOR_GENERATOR;
const CRC32C_EXCEPTION_MESSAGES = {
INVALID_INIT_BASE64_RANGE: (l) => `base64-encoded data expected to equal 4 bytes, not ${l}`,
INVALID_INIT_BUFFER_LENGTH: (l) => `Buffer expected to equal 4 bytes, not ${l}`,
INVALID_INIT_INTEGER: (l) => `Number expected to be a safe, unsigned 32-bit integer, not ${l}`,
};
exports.CRC32C_EXCEPTION_MESSAGES = CRC32C_EXCEPTION_MESSAGES;
class CRC32C {
/**
* Constructs a new `CRC32C` object.
*
* Reconstruction is recommended via the `CRC32C.from` static method.
*
* @param initialValue An initial CRC32C value - a signed 32-bit integer.
*/
constructor(initialValue = 0) {
/** Current CRC32C value */
_CRC32C_crc32c.set(this, 0);
__classPrivateFieldSet(this, _CRC32C_crc32c, initialValue, "f");
}
/**
* Calculates a CRC32C from a provided buffer.
*
* Implementation inspired from:
* - {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc github.com/google/crc32c}
* - {@link https://github.com/googleapis/python-crc32c/blob/a595e758c08df445a99c3bf132ee8e80a3ec4308/src/google_crc32c/python.py github.com/googleapis/python-crc32c}
* - {@link https://github.com/googleapis/java-storage/pull/1376/files github.com/googleapis/java-storage}
*
* @param data The `Buffer` to generate the CRC32C from
*/
update(data) {
let current = __classPrivateFieldGet(this, _CRC32C_crc32c, "f") ^ 0xffffffff;
for (const d of data) {
const tablePoly = CRC32C.CRC32C_EXTENSION_TABLE[(d ^ current) & 0xff];
current = tablePoly ^ (current >>> 8);
}
__classPrivateFieldSet(this, _CRC32C_crc32c, current ^ 0xffffffff, "f");
}
/**
* Validates a provided input to the current CRC32C value.
*
* @param input A Buffer, `CRC32C`-compatible object, base64-encoded data (string), or signed 32-bit integer
*/
validate(input) {
if (typeof input === 'number') {
return input === __classPrivateFieldGet(this, _CRC32C_crc32c, "f");
}
else if (typeof input === 'string') {
return input === this.toString();
}
else if (Buffer.isBuffer(input)) {
return Buffer.compare(input, this.toBuffer()) === 0;
}
else {
// `CRC32C`-like object
return input.toString() === this.toString();
}
}
/**
* Returns a `Buffer` representation of the CRC32C value
*/
toBuffer() {
const buffer = Buffer.alloc(4);
buffer.writeInt32BE(__classPrivateFieldGet(this, _CRC32C_crc32c, "f"));
return buffer;
}
/**
* Returns a JSON-compatible, base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify `JSON#stringify`}
*/
toJSON() {
return this.toString();
}
/**
* Returns a base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/toString `Object#toString`}
*/
toString() {
return this.toBuffer().toString('base64');
}
/**
* Returns the `number` representation of the CRC32C value as a signed 32-bit integer
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/valueOf `Object#valueOf`}
*/
valueOf() {
return __classPrivateFieldGet(this, _CRC32C_crc32c, "f");
}
/**
* Generates a `CRC32C` from a compatible buffer format.
*
* @param value 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`
*/
static fromBuffer(value) {
let buffer;
if (Buffer.isBuffer(value)) {
buffer = value;
}
else if ('buffer' in value) {
// `ArrayBufferView`
buffer = Buffer.from(value.buffer);
}
else {
// `ArrayBuffer`
buffer = Buffer.from(value);
}
if (buffer.byteLength !== 4) {
throw new RangeError(CRC32C_EXCEPTION_MESSAGES.INVALID_INIT_BUFFER_LENGTH(buffer.byteLength));
}
return new CRC32C(buffer.readInt32BE());
}
static async fromFile(file) {
const crc32c = new CRC32C();
await new Promise((resolve, reject) => {
(0, fs_1.createReadStream)(file)
.on('data', (d) => {
if (typeof d === 'string') {
crc32c.update(Buffer.from(d));
}
else {
crc32c.update(d);
}
})
.on('end', () => resolve())
.on('error', reject);
});
return crc32c;
}
/**
* Generates a `CRC32C` from 4-byte base64-encoded data (string).
*
* @param value 4-byte base64-encoded data (string)
*/
static fromString(value) {
const buffer = Buffer.from(value, 'base64');
if (buffer.byteLength !== 4) {
throw new RangeError(CRC32C_EXCEPTION_MESSAGES.INVALID_INIT_BASE64_RANGE(buffer.byteLength));
}
return this.fromBuffer(buffer);
}
/**
* Generates a `CRC32C` from a safe, unsigned 32-bit integer.
*
* @param value an unsigned 32-bit integer
*/
static fromNumber(value) {
if (!Number.isSafeInteger(value) || value > 2 ** 32 || value < -(2 ** 32)) {
throw new RangeError(CRC32C_EXCEPTION_MESSAGES.INVALID_INIT_INTEGER(value));
}
return new CRC32C(value);
}
/**
* Generates a `CRC32C` from a variety of compatable types.
* Note: strings are treated as input, not as file paths to read from.
*
* @param value A number, 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`, or 4-byte base64-encoded data (string)
*/
static from(value) {
if (typeof value === 'number') {
return this.fromNumber(value);
}
else if (typeof value === 'string') {
return this.fromString(value);
}
else if ('byteLength' in value) {
// `ArrayBuffer` | `Buffer` | `ArrayBufferView`
return this.fromBuffer(value);
}
else {
// `CRC32CValidator`/`CRC32C`-like
return this.fromString(value.toString());
}
}
}
exports.CRC32C = CRC32C;
_CRC32C_crc32c = new WeakMap();
CRC32C.CRC32C_EXTENSIONS = CRC32C_EXTENSIONS;
CRC32C.CRC32C_EXTENSION_TABLE = CRC32C_EXTENSION_TABLE;

View File

@@ -0,0 +1,982 @@
import { BodyResponseCallback, DecorateRequestOptions, GetConfig, MetadataCallback, ServiceObject, SetMetadataResponse } from './nodejs-common/index.js';
import * as resumableUpload from './resumable-upload.js';
import { Writable, Readable, PipelineSource } from 'stream';
import * as http from 'http';
import { PreconditionOptions, Storage } from './storage.js';
import { AvailableServiceObjectMethods, Bucket } from './bucket.js';
import { Acl, AclMetadata } from './acl.js';
import { GetSignedUrlResponse, GetSignedUrlCallback, URLSigner, SignerGetSignedUrlConfig, Query } from './signer.js';
import { Duplexify, GCCL_GCS_CMD_KEY } from './nodejs-common/util.js';
import { CRC32C, CRC32CValidatorGenerator } from './crc32c.js';
import { URL } from 'url';
import { BaseMetadata, DeleteCallback, DeleteOptions, GetResponse, InstanceResponseCallback, RequestResponse, SetMetadataOptions } from './nodejs-common/service-object.js';
import * as r from 'teeny-request';
export type GetExpirationDateResponse = [Date];
export interface GetExpirationDateCallback {
(err: Error | null, expirationDate?: Date | null, apiResponse?: unknown): void;
}
export interface PolicyDocument {
string: string;
base64: string;
signature: string;
}
export type SaveData = string | Buffer | Uint8Array | PipelineSource<string | Buffer | Uint8Array>;
export type GenerateSignedPostPolicyV2Response = [PolicyDocument];
export interface GenerateSignedPostPolicyV2Callback {
(err: Error | null, policy?: PolicyDocument): void;
}
export interface GenerateSignedPostPolicyV2Options {
equals?: string[] | string[][];
expires: string | number | Date;
startsWith?: string[] | string[][];
acl?: string;
successRedirect?: string;
successStatus?: string;
contentLengthRange?: {
min?: number;
max?: number;
};
/**
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string;
}
export interface PolicyFields {
[key: string]: string;
}
export interface GenerateSignedPostPolicyV4Options {
expires: string | number | Date;
bucketBoundHostname?: string;
virtualHostedStyle?: boolean;
conditions?: object[];
fields?: PolicyFields;
/**
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string;
}
export interface GenerateSignedPostPolicyV4Callback {
(err: Error | null, output?: SignedPostPolicyV4Output): void;
}
export type GenerateSignedPostPolicyV4Response = [SignedPostPolicyV4Output];
export interface SignedPostPolicyV4Output {
url: string;
fields: PolicyFields;
}
export interface GetSignedUrlConfig extends Pick<SignerGetSignedUrlConfig, 'host' | 'signingEndpoint'> {
action: 'read' | 'write' | 'delete' | 'resumable';
version?: 'v2' | 'v4';
virtualHostedStyle?: boolean;
cname?: string;
contentMd5?: string;
contentType?: string;
expires: string | number | Date;
accessibleAt?: string | number | Date;
extensionHeaders?: http.OutgoingHttpHeaders;
promptSaveAs?: string;
responseDisposition?: string;
responseType?: string;
queryParams?: Query;
}
export interface GetFileMetadataOptions {
userProject?: string;
}
export type GetFileMetadataResponse = [FileMetadata, unknown];
export interface GetFileMetadataCallback {
(err: Error | null, metadata?: FileMetadata, apiResponse?: unknown): void;
}
export interface GetFileOptions extends GetConfig {
userProject?: string;
generation?: number;
restoreToken?: string;
softDeleted?: boolean;
}
export type GetFileResponse = [File, unknown];
export interface GetFileCallback {
(err: Error | null, file?: File, apiResponse?: unknown): void;
}
export interface FileExistsOptions {
userProject?: string;
}
export type FileExistsResponse = [boolean];
export interface FileExistsCallback {
(err: Error | null, exists?: boolean): void;
}
export interface DeleteFileOptions {
ignoreNotFound?: boolean;
userProject?: string;
}
export type DeleteFileResponse = [unknown];
export interface DeleteFileCallback {
(err: Error | null, apiResponse?: unknown): void;
}
export type PredefinedAcl = 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
type PublicResumableUploadOptions = 'chunkSize' | 'highWaterMark' | 'isPartialUpload' | 'metadata' | 'origin' | 'offset' | 'predefinedAcl' | 'private' | 'public' | 'uri' | 'userProject';
export interface CreateResumableUploadOptions extends Pick<resumableUpload.UploadConfig, PublicResumableUploadOptions> {
/**
* A CRC32C to resume from when continuing a previous upload. It is recommended
* to capture the `crc32c` event from previous upload sessions to provide in
* subsequent requests in order to accurately track the upload. This is **required**
* when validating a final portion of the uploaded object.
*
* @see {@link CRC32C.from} for possible values.
*/
resumeCRC32C?: Parameters<(typeof CRC32C)['from']>[0];
preconditionOpts?: PreconditionOptions;
[GCCL_GCS_CMD_KEY]?: resumableUpload.UploadConfig[typeof GCCL_GCS_CMD_KEY];
}
export type CreateResumableUploadResponse = [string];
export interface CreateResumableUploadCallback {
(err: Error | null, uri?: string): void;
}
export interface CreateWriteStreamOptions extends CreateResumableUploadOptions {
contentType?: string;
gzip?: string | boolean;
resumable?: boolean;
timeout?: number;
validation?: string | boolean;
}
export interface MakeFilePrivateOptions {
metadata?: FileMetadata;
strict?: boolean;
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type MakeFilePrivateResponse = [unknown];
export type MakeFilePrivateCallback = SetFileMetadataCallback;
export interface IsPublicCallback {
(err: Error | null, resp?: boolean): void;
}
export type IsPublicResponse = [boolean];
export type MakeFilePublicResponse = [unknown];
export interface MakeFilePublicCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type MoveResponse = [unknown];
export interface MoveCallback {
(err: Error | null, destinationFile?: File | null, apiResponse?: unknown): void;
}
export interface MoveOptions {
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type MoveFileAtomicOptions = MoveOptions;
export type MoveFileAtomicCallback = MoveCallback;
export type MoveFileAtomicResponse = MoveResponse;
export type RenameOptions = MoveOptions;
export type RenameResponse = MoveResponse;
export type RenameCallback = MoveCallback;
export type RotateEncryptionKeyOptions = string | Buffer | EncryptionKeyOptions;
export interface EncryptionKeyOptions {
encryptionKey?: string | Buffer;
kmsKeyName?: string;
preconditionOpts?: PreconditionOptions;
}
export type RotateEncryptionKeyCallback = CopyCallback;
export type RotateEncryptionKeyResponse = CopyResponse;
export declare enum ActionToHTTPMethod {
read = "GET",
write = "PUT",
delete = "DELETE",
resumable = "POST"
}
/**
* @deprecated - no longer used
*/
export declare const STORAGE_POST_POLICY_BASE_URL = "https://storage.googleapis.com";
export interface FileOptions {
crc32cGenerator?: CRC32CValidatorGenerator;
encryptionKey?: string | Buffer;
generation?: number | string;
restoreToken?: string;
kmsKeyName?: string;
preconditionOpts?: PreconditionOptions;
userProject?: string;
}
export interface CopyOptions {
cacheControl?: string;
contentEncoding?: string;
contentType?: string;
contentDisposition?: string;
destinationKmsKeyName?: string;
metadata?: {
[key: string]: string | boolean | number | null;
};
predefinedAcl?: string;
token?: string;
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type CopyResponse = [File, unknown];
export interface CopyCallback {
(err: Error | null, file?: File | null, apiResponse?: unknown): void;
}
export type DownloadResponse = [Buffer];
export type DownloadCallback = (err: RequestError | null, contents: Buffer) => void;
export interface DownloadOptions extends CreateReadStreamOptions {
destination?: string;
encryptionKey?: string | Buffer;
}
export interface CreateReadStreamOptions {
userProject?: string;
validation?: 'md5' | 'crc32c' | false | true;
start?: number;
end?: number;
decompress?: boolean;
[GCCL_GCS_CMD_KEY]?: string;
}
export interface SaveOptions extends CreateWriteStreamOptions {
onUploadProgress?: (progressEvent: any) => void;
}
export interface SaveCallback {
(err?: Error | null): void;
}
export interface SetFileMetadataOptions {
userProject?: string;
}
export interface SetFileMetadataCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type SetFileMetadataResponse = [unknown];
export type SetStorageClassResponse = [unknown];
export interface SetStorageClassOptions {
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export interface SetStorageClassCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export interface RestoreOptions extends PreconditionOptions {
generation: number;
restoreToken?: string;
projection?: 'full' | 'noAcl';
}
export interface FileMetadata extends BaseMetadata {
acl?: AclMetadata[] | null;
bucket?: string;
cacheControl?: string;
componentCount?: number;
contentDisposition?: string;
contentEncoding?: string;
contentLanguage?: string;
contentType?: string;
crc32c?: string;
customerEncryption?: {
encryptionAlgorithm?: string;
keySha256?: string;
};
customTime?: string;
eventBasedHold?: boolean | null;
readonly eventBasedHoldReleaseTime?: string;
generation?: string | number;
restoreToken?: string;
hardDeleteTime?: string;
kmsKeyName?: string;
md5Hash?: string;
mediaLink?: string;
metadata?: {
[key: string]: string | boolean | number | null;
};
metageneration?: string | number;
name?: string;
owner?: {
entity?: string;
entityId?: string;
};
retention?: {
retainUntilTime?: string;
mode?: string;
} | null;
retentionExpirationTime?: string;
size?: string | number;
softDeleteTime?: string;
storageClass?: string;
temporaryHold?: boolean | null;
timeCreated?: string;
timeDeleted?: string;
timeStorageClassUpdated?: string;
updated?: string;
}
export declare class RequestError extends Error {
code?: string;
errors?: Error[];
}
export declare enum FileExceptionMessages {
EXPIRATION_TIME_NA = "An expiration time is not available.",
DESTINATION_NO_NAME = "Destination file should have a name.",
INVALID_VALIDATION_FILE_RANGE = "Cannot use validation with file ranges (start/end).",
MD5_NOT_AVAILABLE = "MD5 verification was specified, but is not available for the requested object. MD5 is not available for composite objects.",
EQUALS_CONDITION_TWO_ELEMENTS = "Equals condition must be an array of 2 elements.",
STARTS_WITH_TWO_ELEMENTS = "StartsWith condition must be an array of 2 elements.",
CONTENT_LENGTH_RANGE_MIN_MAX = "ContentLengthRange must have numeric min & max fields.",
DOWNLOAD_MISMATCH = "The downloaded data did not match the data from the server. To be sure the content is the same, you should download the file again.",
UPLOAD_MISMATCH_DELETE_FAIL = "The uploaded data did not match the data from the server.\n As a precaution, we attempted to delete the file, but it was not successful.\n To be sure the content is the same, you should try removing the file manually,\n then uploading the file again.\n \n\nThe delete attempt failed with this message:\n\n ",
UPLOAD_MISMATCH = "The uploaded data did not match the data from the server.\n As a precaution, the file has been deleted.\n To be sure the content is the same, you should try uploading the file again.",
MD5_RESUMED_UPLOAD = "MD5 cannot be used with a continued resumable upload as MD5 cannot be extended from an existing value",
MISSING_RESUME_CRC32C_FINAL_UPLOAD = "The CRC32C is missing for the final portion of a resumed upload, which is required for validation. Please provide `resumeCRC32C` if validation is required, or disable `validation`."
}
/**
* A File object is created from your {@link Bucket} object using
* {@link Bucket#file}.
*
* @class
*/
declare class File extends ServiceObject<File, FileMetadata> {
#private;
acl: Acl;
crc32cGenerator: CRC32CValidatorGenerator;
bucket: Bucket;
storage: Storage;
kmsKeyName?: string;
userProject?: string;
signer?: URLSigner;
name: string;
generation?: number;
restoreToken?: string;
parent: Bucket;
private encryptionKey?;
private encryptionKeyBase64?;
private encryptionKeyHash?;
private encryptionKeyInterceptor?;
private instanceRetryValue?;
instancePreconditionOpts?: PreconditionOptions;
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against
* an object or bucket (for example, `READ` or `WRITE`); the entity defines
* who the permission applies to (for example, a specific user or group of
* users).
*
* The `acl` object on a File instance provides methods to get you a list of
* the ACLs defined on your bucket, as well as set, update, and delete them.
*
* See {@link http://goo.gl/6qBBPO| About Access Control lists}
*
* @name File#acl
* @mixes Acl
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
* //-
* // Make a file publicly readable.
* //-
* const options = {
* entity: 'allUsers',
* role: storage.acl.READER_ROLE
* };
*
* file.acl.add(options, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* file.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
/**
* The API-formatted resource description of the file.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name File#metadata
* @type {object}
*/
/**
* The file's name.
* @name File#name
* @type {string}
*/
/**
* @callback Crc32cGeneratorToStringCallback
* A method returning the CRC32C as a base64-encoded string.
*
* @returns {string}
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
/**
* @callback Crc32cGeneratorValidateCallback
* A method validating a base64-encoded CRC32C string.
*
* @param {string} [value] base64-encoded CRC32C string to validate
* @returns {boolean}
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
**/
/**
* @callback Crc32cGeneratorUpdateCallback
* A method for passing `Buffer`s for CRC32C generation.
*
* @param {Buffer} [data] data to update CRC32C value with
* @returns {undefined}
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
**/
/**
* @typedef {object} CRC32CValidator
* @property {Crc32cGeneratorToStringCallback}
* @property {Crc32cGeneratorValidateCallback}
* @property {Crc32cGeneratorUpdateCallback}
*/
/**
* @callback Crc32cGeneratorCallback
* @returns {CRC32CValidator}
*/
/**
* @typedef {object} FileOptions Options passed to the File constructor.
* @property {string} [encryptionKey] A custom encryption key.
* @property {number} [generation] Generation to scope the file to.
* @property {string} [kmsKeyName] Cloud KMS Key used to encrypt this
* object, if the object is encrypted by such a key. Limited availability;
* usable only by enabled projects.
* @property {string} [userProject] The ID of the project which will be
* billed for all requests made from File object.
* @property {Crc32cGeneratorCallback} [callback] A function that generates a CRC32C Validator. Defaults to {@link CRC32C}
*/
/**
* Constructs a file object.
*
* @param {Bucket} bucket The Bucket instance this file is
* attached to.
* @param {string} name The name of the remote file.
* @param {FileOptions} [options] Configuration options.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
* ```
*/
constructor(bucket: Bucket, name: string, options?: FileOptions);
/**
* The object's Cloud Storage URI (`gs://`)
*
* @example
* ```ts
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const file = bucket.file('image.png');
*
* // `gs://my-bucket/image.png`
* const href = file.cloudStorageURI.href;
* ```
*/
get cloudStorageURI(): URL;
/**
* A helper method for determining if a request should be retried based on preconditions.
* This should only be used for methods where the idempotency is determined by
* `ifGenerationMatch`
* @private
*
* A request should not be retried under the following conditions:
* - if precondition option `ifGenerationMatch` is not set OR
* - if `idempotencyStrategy` is set to `RetryNever`
*/
private shouldRetryBasedOnPreconditionAndIdempotencyStrat;
copy(destination: string | Bucket | File, options?: CopyOptions): Promise<CopyResponse>;
copy(destination: string | Bucket | File, callback: CopyCallback): void;
copy(destination: string | Bucket | File, options: CopyOptions, callback: CopyCallback): void;
/**
* @typedef {object} CreateReadStreamOptions Configuration options for File#createReadStream.
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
* @property {string|boolean} [validation] Possible values: `"md5"`,
* `"crc32c"`, or `false`. By default, data integrity is validated with a
* CRC32c checksum. You may use MD5 if preferred, but that hash is not
* supported for composite objects. An error will be raised if MD5 is
* specified but is not available. You may also choose to skip validation
* completely, however this is **not recommended**.
* @property {number} [start] A byte offset to begin the file's download
* from. Default is 0. NOTE: Byte ranges are inclusive; that is,
* `options.start = 0` and `options.end = 999` represent the first 1000
* bytes in a file or object. NOTE: when specifying a byte range, data
* integrity is not available.
* @property {number} [end] A byte offset to stop reading the file at.
* NOTE: Byte ranges are inclusive; that is, `options.start = 0` and
* `options.end = 999` represent the first 1000 bytes in a file or object.
* NOTE: when specifying a byte range, data integrity is not available.
* @property {boolean} [decompress=true] Disable auto decompression of the
* received data. By default this option is set to `true`.
* Applicable in cases where the data was uploaded with
* `gzip: true` option. See {@link File#createWriteStream}.
*/
/**
* Create a readable stream to read the contents of the remote file. It can be
* piped to a writable stream or listened to for 'data' events to read a
* file's contents.
*
* In the unlikely event there is a mismatch between what you downloaded and
* the version in your Bucket, your error handler will receive an error with
* code "CONTENT_DOWNLOAD_MISMATCH". If you receive this error, the best
* recourse is to try downloading the file again.
*
* NOTE: Readable streams will emit the `end` event when the file is fully
* downloaded.
*
* @param {CreateReadStreamOptions} [options] Configuration options.
* @returns {ReadableStream}
*
* @example
* ```
* //-
* // <h4>Downloading a File</h4>
* //
* // The example below demonstrates how we can reference a remote file, then
* // pipe its contents to a local file. This is effectively creating a local
* // backup of your remote data.
* //-
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* const fs = require('fs');
* const remoteFile = bucket.file('image.png');
* const localFilename = '/Users/stephen/Photos/image.png';
*
* remoteFile.createReadStream()
* .on('error', function(err) {})
* .on('response', function(response) {
* // Server connected and responded with the specified status and headers.
* })
* .on('end', function() {
* // The file is fully downloaded.
* })
* .pipe(fs.createWriteStream(localFilename));
*
* //-
* // To limit the downloaded data to only a byte range, pass an options
* // object.
* //-
* const logFile = myBucket.file('access_log');
* logFile.createReadStream({
* start: 10000,
* end: 20000
* })
* .on('error', function(err) {})
* .pipe(fs.createWriteStream('/Users/stephen/logfile.txt'));
*
* //-
* // To read a tail byte range, specify only `options.end` as a negative
* // number.
* //-
* const logFile = myBucket.file('access_log');
* logFile.createReadStream({
* end: -100
* })
* .on('error', function(err) {})
* .pipe(fs.createWriteStream('/Users/stephen/logfile.txt'));
* ```
*/
createReadStream(options?: CreateReadStreamOptions): Readable;
createResumableUpload(options?: CreateResumableUploadOptions): Promise<CreateResumableUploadResponse>;
createResumableUpload(options: CreateResumableUploadOptions, callback: CreateResumableUploadCallback): void;
createResumableUpload(callback: CreateResumableUploadCallback): void;
/**
* @typedef {object} CreateWriteStreamOptions Configuration options for File#createWriteStream().
* @property {string} [contentType] Alias for
* `options.metadata.contentType`. If set to `auto`, the file name is used
* to determine the contentType.
* @property {string|boolean} [gzip] If true, automatically gzip the file.
* If set to `auto`, the contentType is used to determine if the file
* should be gzipped. This will set `options.metadata.contentEncoding` to
* `gzip` if necessary.
* @property {object} [metadata] See the examples below or
* {@link https://cloud.google.com/storage/docs/json_api/v1/objects/insert#request_properties_JSON| Objects: insert request body}
* for more details.
* @property {number} [offset] The starting byte of the upload stream, for
* resuming an interrupted upload. Defaults to 0.
* @property {string} [predefinedAcl] Apply a predefined set of access
* controls to this object.
*
* Acceptable values are:
* - **`authenticatedRead`** - Object owner gets `OWNER` access, and
* `allAuthenticatedUsers` get `READER` access.
*
* - **`bucketOwnerFullControl`** - Object owner gets `OWNER` access, and
* project team owners get `OWNER` access.
*
* - **`bucketOwnerRead`** - Object owner gets `OWNER` access, and project
* team owners get `READER` access.
*
* - **`private`** - Object owner gets `OWNER` access.
*
* - **`projectPrivate`** - Object owner gets `OWNER` access, and project
* team members get access according to their roles.
*
* - **`publicRead`** - Object owner gets `OWNER` access, and `allUsers`
* get `READER` access.
* @property {boolean} [private] Make the uploaded file private. (Alias for
* `options.predefinedAcl = 'private'`)
* @property {boolean} [public] Make the uploaded file public. (Alias for
* `options.predefinedAcl = 'publicRead'`)
* @property {boolean} [resumable] Force a resumable upload. NOTE: When
* working with streams, the file format and size is unknown until it's
* completely consumed. Because of this, it's best for you to be explicit
* for what makes sense given your input.
* @property {number} [timeout=60000] Set the HTTP request timeout in
* milliseconds. This option is not available for resumable uploads.
* Default: `60000`
* @property {string} [uri] The URI for an already-created resumable
* upload. See {@link File#createResumableUpload}.
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
* @property {string|boolean} [validation] Possible values: `"md5"`,
* `"crc32c"`, or `false`. By default, data integrity is validated with a
* CRC32c checksum. You may use MD5 if preferred, but that hash is not
* supported for composite objects. An error will be raised if MD5 is
* specified but is not available. You may also choose to skip validation
* completely, however this is **not recommended**. In addition to specifying
* validation type, providing `metadata.crc32c` or `metadata.md5Hash` will
* cause the server to perform validation in addition to client validation.
* NOTE: Validation is automatically skipped for objects that were
* uploaded using the `gzip` option and have already compressed content.
*/
/**
* Create a writable stream to overwrite the contents of the file in your
* bucket.
*
* A File object can also be used to create files for the first time.
*
* Resumable uploads are automatically enabled and must be shut off explicitly
* by setting `options.resumable` to `false`.
*
*
* <p class="notice">
* There is some overhead when using a resumable upload that can cause
* noticeable performance degradation while uploading a series of small
* files. When uploading files less than 10MB, it is recommended that the
* resumable feature is disabled.
* </p>
*
* NOTE: Writable streams will emit the `finish` event when the file is fully
* uploaded.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload Upload Options (Simple or Resumable)}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objects/insert Objects: insert API Documentation}
*
* @param {CreateWriteStreamOptions} [options] Configuration options.
* @returns {WritableStream}
*
* @example
* ```
* const fs = require('fs');
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
*
* //-
* // <h4>Uploading a File</h4>
* //
* // Now, consider a case where we want to upload a file to your bucket. You
* // have the option of using {@link Bucket#upload}, but that is just
* // a convenience method which will do the following.
* //-
* fs.createReadStream('/Users/stephen/Photos/birthday-at-the-zoo/panda.jpg')
* .pipe(file.createWriteStream())
* .on('error', function(err) {})
* .on('finish', function() {
* // The file upload is complete.
* });
*
* //-
* // <h4>Uploading a File with gzip compression</h4>
* //-
* fs.createReadStream('/Users/stephen/site/index.html')
* .pipe(file.createWriteStream({ gzip: true }))
* .on('error', function(err) {})
* .on('finish', function() {
* // The file upload is complete.
* });
*
* //-
* // Downloading the file with `createReadStream` will automatically decode
* // the file.
* //-
*
* //-
* // <h4>Uploading a File with Metadata</h4>
* //
* // One last case you may run into is when you want to upload a file to your
* // bucket and set its metadata at the same time. Like above, you can use
* // {@link Bucket#upload} to do this, which is just a wrapper around
* // the following.
* //-
* fs.createReadStream('/Users/stephen/Photos/birthday-at-the-zoo/panda.jpg')
* .pipe(file.createWriteStream({
* metadata: {
* contentType: 'image/jpeg',
* metadata: {
* custom: 'metadata'
* }
* }
* }))
* .on('error', function(err) {})
* .on('finish', function() {
* // The file upload is complete.
* });
* ```
*
* //-
* // <h4>Continuing a Resumable Upload</h4>
* //
* // One can capture a `uri` from a resumable upload to reuse later.
* // Additionally, for validation, one can also capture and pass `crc32c`.
* //-
* let uri: string | undefined = undefined;
* let resumeCRC32C: string | undefined = undefined;
*
* fs.createWriteStream()
* .on('uri', link => {uri = link})
* .on('crc32', crc32c => {resumeCRC32C = crc32c});
*
* // later...
* fs.createWriteStream({uri, resumeCRC32C});
*/
createWriteStream(options?: CreateWriteStreamOptions): Writable;
/**
* Delete the object.
*
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
delete(options?: DeleteOptions): Promise<[r.Response]>;
delete(options: DeleteOptions, callback: DeleteCallback): void;
delete(callback: DeleteCallback): void;
download(options?: DownloadOptions): Promise<DownloadResponse>;
download(options: DownloadOptions, callback: DownloadCallback): void;
download(callback: DownloadCallback): void;
/**
* The Storage API allows you to use a custom key for server-side encryption.
*
* See {@link https://cloud.google.com/storage/docs/encryption#customer-supplied| Customer-supplied Encryption Keys}
*
* @param {string|buffer} encryptionKey An AES-256 encryption key.
* @returns {File}
*
* @example
* ```
* const crypto = require('crypto');
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const encryptionKey = crypto.randomBytes(32);
*
* const fileWithCustomEncryption = myBucket.file('my-file');
* fileWithCustomEncryption.setEncryptionKey(encryptionKey);
*
* const fileWithoutCustomEncryption = myBucket.file('my-file');
*
* fileWithCustomEncryption.save('data', function(err) {
* // Try to download with the File object that hasn't had
* // `setEncryptionKey()` called:
* fileWithoutCustomEncryption.download(function(err) {
* // We will receive an error:
* // err.message === 'Bad Request'
*
* // Try again with the File object we called `setEncryptionKey()` on:
* fileWithCustomEncryption.download(function(err, contents) {
* // contents.toString() === 'data'
* });
* });
* });
*
* ```
* @example <caption>include:samples/encryption.js</caption>
* region_tag:storage_upload_encrypted_file
* Example of uploading an encrypted file:
*
* @example <caption>include:samples/encryption.js</caption>
* region_tag:storage_download_encrypted_file
* Example of downloading an encrypted file:
*/
setEncryptionKey(encryptionKey: string | Buffer): this;
/**
* Gets a reference to a Cloud Storage {@link File} file from the provided URL in string format.
* @param {string} publicUrlOrGsUrl the URL as a string. Must be of the format gs://bucket/file
* or https://storage.googleapis.com/bucket/file.
* @param {Storage} storageInstance an instance of a Storage object.
* @param {FileOptions} [options] Configuration options
* @returns {File}
*/
static from(publicUrlOrGsUrl: string, storageInstance: Storage, options?: FileOptions): File;
get(options?: GetFileOptions): Promise<GetResponse<File>>;
get(callback: InstanceResponseCallback<File>): void;
get(options: GetFileOptions, callback: InstanceResponseCallback<File>): void;
getExpirationDate(): Promise<GetExpirationDateResponse>;
getExpirationDate(callback: GetExpirationDateCallback): void;
generateSignedPostPolicyV2(options: GenerateSignedPostPolicyV2Options): Promise<GenerateSignedPostPolicyV2Response>;
generateSignedPostPolicyV2(options: GenerateSignedPostPolicyV2Options, callback: GenerateSignedPostPolicyV2Callback): void;
generateSignedPostPolicyV2(callback: GenerateSignedPostPolicyV2Callback): void;
generateSignedPostPolicyV4(options: GenerateSignedPostPolicyV4Options): Promise<GenerateSignedPostPolicyV4Response>;
generateSignedPostPolicyV4(options: GenerateSignedPostPolicyV4Options, callback: GenerateSignedPostPolicyV4Callback): void;
generateSignedPostPolicyV4(callback: GenerateSignedPostPolicyV4Callback): void;
getSignedUrl(cfg: GetSignedUrlConfig): Promise<GetSignedUrlResponse>;
getSignedUrl(cfg: GetSignedUrlConfig, callback: GetSignedUrlCallback): void;
isPublic(): Promise<IsPublicResponse>;
isPublic(callback: IsPublicCallback): void;
makePrivate(options?: MakeFilePrivateOptions): Promise<MakeFilePrivateResponse>;
makePrivate(callback: MakeFilePrivateCallback): void;
makePrivate(options: MakeFilePrivateOptions, callback: MakeFilePrivateCallback): void;
makePublic(): Promise<MakeFilePublicResponse>;
makePublic(callback: MakeFilePublicCallback): void;
/**
* The public URL of this File
* Use {@link File#makePublic} to enable anonymous access via the returned URL.
*
* @returns {string}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* const file = bucket.file('my-file');
*
* // publicUrl will be "https://storage.googleapis.com/albums/my-file"
* const publicUrl = file.publicUrl();
* ```
*/
publicUrl(): string;
moveFileAtomic(destination: string | File, options?: MoveFileAtomicOptions): Promise<MoveFileAtomicResponse>;
moveFileAtomic(destination: string | File, callback: MoveFileAtomicCallback): void;
moveFileAtomic(destination: string | File, options: MoveFileAtomicOptions, callback: MoveFileAtomicCallback): void;
move(destination: string | Bucket | File, options?: MoveOptions): Promise<MoveResponse>;
move(destination: string | Bucket | File, callback: MoveCallback): void;
move(destination: string | Bucket | File, options: MoveOptions, callback: MoveCallback): void;
rename(destinationFile: string | File, options?: RenameOptions): Promise<RenameResponse>;
rename(destinationFile: string | File, callback: RenameCallback): void;
rename(destinationFile: string | File, options: RenameOptions, callback: RenameCallback): void;
/**
* @typedef {object} RestoreOptions Options for File#restore(). See an
* {@link https://cloud.google.com/storage/docs/json_api/v1/objects#resource| Object resource}.
* @param {string} [userProject] The ID of the project which will be
* billed for the request.
* @param {number} [generation] If present, selects a specific revision of this object.
* @param {string} [restoreToken] Returns an option that must be specified when getting a soft-deleted object from an HNS-enabled
* bucket that has a naming and generation conflict with another object in the same bucket.
* @param {string} [projection] Specifies the set of properties to return. If used, must be 'full' or 'noAcl'.
* @param {string | number} [ifGenerationMatch] Request proceeds if the generation of the target resource
* matches the value used in the precondition.
* If the values don't match, the request fails with a 412 Precondition Failed response.
* @param {string | number} [ifGenerationNotMatch] Request proceeds if the generation of the target resource does
* not match the value used in the precondition. If the values match, the request fails with a 304 Not Modified response.
* @param {string | number} [ifMetagenerationMatch] Request proceeds if the meta-generation of the target resource
* matches the value used in the precondition.
* If the values don't match, the request fails with a 412 Precondition Failed response.
* @param {string | number} [ifMetagenerationNotMatch] Request proceeds if the meta-generation of the target resource does
* not match the value used in the precondition. If the values match, the request fails with a 304 Not Modified response.
*/
/**
* Restores a soft-deleted file
* @param {RestoreOptions} options Restore options.
* @returns {Promise<File>}
*/
restore(options: RestoreOptions): Promise<File>;
request(reqOpts: DecorateRequestOptions): Promise<RequestResponse>;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
rotateEncryptionKey(options?: RotateEncryptionKeyOptions): Promise<RotateEncryptionKeyResponse>;
rotateEncryptionKey(callback: RotateEncryptionKeyCallback): void;
rotateEncryptionKey(options: RotateEncryptionKeyOptions, callback: RotateEncryptionKeyCallback): void;
save(data: SaveData, options?: SaveOptions): Promise<void>;
save(data: SaveData, callback: SaveCallback): void;
save(data: SaveData, options: SaveOptions, callback: SaveCallback): void;
setMetadata(metadata: FileMetadata, options?: SetMetadataOptions): Promise<SetMetadataResponse<FileMetadata>>;
setMetadata(metadata: FileMetadata, callback: MetadataCallback<FileMetadata>): void;
setMetadata(metadata: FileMetadata, options: SetMetadataOptions, callback: MetadataCallback<FileMetadata>): void;
setStorageClass(storageClass: string, options?: SetStorageClassOptions): Promise<SetStorageClassResponse>;
setStorageClass(storageClass: string, options: SetStorageClassOptions, callback: SetStorageClassCallback): void;
setStorageClass(storageClass: string, callback?: SetStorageClassCallback): void;
/**
* Set a user project to be billed for all requests made from this File
* object.
*
* @param {string} userProject The user project.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* const file = bucket.file('my-file');
*
* file.setUserProject('grape-spaceship-123');
* ```
*/
setUserProject(userProject: string): void;
/**
* This creates a resumable-upload upload stream.
*
* @param {Duplexify} stream - Duplexify stream of data to pipe to the file.
* @param {object=} options - Configuration object.
*
* @private
*/
startResumableUpload_(dup: Duplexify, options?: CreateResumableUploadOptions): void;
/**
* Takes a readable stream and pipes it to a remote file. Unlike
* `startResumableUpload_`, which uses the resumable upload technique, this
* method uses a simple upload (all or nothing).
*
* @param {Duplexify} dup - Duplexify stream of data to pipe to the file.
* @param {object=} options - Configuration object.
*
* @private
*/
startSimpleUpload_(dup: Duplexify, options?: CreateWriteStreamOptions): void;
disableAutoRetryConditionallyIdempotent_(coreOpts: any, methodType: AvailableServiceObjectMethods, localPreconditionOptions?: PreconditionOptions): void;
private getBufferFromReadable;
}
/**
* Reference to the {@link File} class.
* @name module:@google-cloud/storage.File
* @see File
*/
export { File };

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,35 @@
import { Transform } from 'stream';
import { CRC32CValidatorGenerator, CRC32CValidator } from './crc32c.js';
interface HashStreamValidatorOptions {
/** Enables CRC32C calculation. To validate a provided value use `crc32cExpected`. */
crc32c: boolean;
/** Enables MD5 calculation. To validate a provided value use `md5Expected`. */
md5: boolean;
/** A CRC32C instance for validation. To validate a provided value use `crc32cExpected`. */
crc32cInstance: CRC32CValidator;
/** Set a custom CRC32C generator. Used if `crc32cInstance` has not been provided. */
crc32cGenerator: CRC32CValidatorGenerator;
/** Sets the expected CRC32C value to verify once all data has been consumed. Also sets the `crc32c` option to `true` */
crc32cExpected?: string;
/** Sets the expected MD5 value to verify once all data has been consumed. Also sets the `md5` option to `true` */
md5Expected?: string;
/** Indicates whether or not to run a validation check or only update the hash values */
updateHashesOnly?: boolean;
}
declare class HashStreamValidator extends Transform {
#private;
readonly crc32cEnabled: boolean;
readonly md5Enabled: boolean;
readonly crc32cExpected: string | undefined;
readonly md5Expected: string | undefined;
readonly updateHashesOnly: boolean;
constructor(options?: Partial<HashStreamValidatorOptions>);
/**
* Return the current CRC32C value, if available.
*/
get crc32c(): string | undefined;
_flush(callback: (error?: Error | null | undefined) => void): void;
_transform(chunk: Buffer, encoding: BufferEncoding, callback: (e?: Error) => void): void;
test(hash: 'crc32c' | 'md5', sum: Buffer | string): boolean;
}
export { HashStreamValidator, HashStreamValidatorOptions };

View File

@@ -0,0 +1,119 @@
"use strict";
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _HashStreamValidator_crc32cHash, _HashStreamValidator_md5Hash, _HashStreamValidator_md5Digest;
Object.defineProperty(exports, "__esModule", { value: true });
exports.HashStreamValidator = void 0;
const crypto_1 = require("crypto");
const stream_1 = require("stream");
const crc32c_js_1 = require("./crc32c.js");
const file_js_1 = require("./file.js");
class HashStreamValidator extends stream_1.Transform {
constructor(options = {}) {
super();
this.updateHashesOnly = false;
_HashStreamValidator_crc32cHash.set(this, undefined);
_HashStreamValidator_md5Hash.set(this, undefined);
_HashStreamValidator_md5Digest.set(this, '');
this.crc32cEnabled = !!options.crc32c;
this.md5Enabled = !!options.md5;
this.updateHashesOnly = !!options.updateHashesOnly;
this.crc32cExpected = options.crc32cExpected;
this.md5Expected = options.md5Expected;
if (this.crc32cEnabled) {
if (options.crc32cInstance) {
__classPrivateFieldSet(this, _HashStreamValidator_crc32cHash, options.crc32cInstance, "f");
}
else {
const crc32cGenerator = options.crc32cGenerator || crc32c_js_1.CRC32C_DEFAULT_VALIDATOR_GENERATOR;
__classPrivateFieldSet(this, _HashStreamValidator_crc32cHash, crc32cGenerator(), "f");
}
}
if (this.md5Enabled) {
__classPrivateFieldSet(this, _HashStreamValidator_md5Hash, (0, crypto_1.createHash)('md5'), "f");
}
}
/**
* Return the current CRC32C value, if available.
*/
get crc32c() {
var _a;
return (_a = __classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f")) === null || _a === void 0 ? void 0 : _a.toString();
}
_flush(callback) {
if (__classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f")) {
__classPrivateFieldSet(this, _HashStreamValidator_md5Digest, __classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f").digest('base64'), "f");
}
if (this.updateHashesOnly) {
callback();
return;
}
// If we're doing validation, assume the worst-- a data integrity
// mismatch. If not, these tests won't be performed, and we can assume
// the best.
// We must check if the server decompressed the data on serve because hash
// validation is not possible in this case.
let failed = this.crc32cEnabled || this.md5Enabled;
if (this.crc32cEnabled && this.crc32cExpected) {
failed = !this.test('crc32c', this.crc32cExpected);
}
if (this.md5Enabled && this.md5Expected) {
failed = !this.test('md5', this.md5Expected);
}
if (failed) {
const mismatchError = new file_js_1.RequestError(file_js_1.FileExceptionMessages.DOWNLOAD_MISMATCH);
mismatchError.code = 'CONTENT_DOWNLOAD_MISMATCH';
callback(mismatchError);
}
else {
callback();
}
}
_transform(chunk, encoding, callback) {
this.push(chunk, encoding);
try {
if (__classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f"))
__classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f").update(chunk);
if (__classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f"))
__classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f").update(chunk);
callback();
}
catch (e) {
callback(e);
}
}
test(hash, sum) {
const check = Buffer.isBuffer(sum) ? sum.toString('base64') : sum;
if (hash === 'crc32c' && __classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f")) {
return __classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f").validate(check);
}
if (hash === 'md5' && __classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f")) {
return __classPrivateFieldGet(this, _HashStreamValidator_md5Digest, "f") === check;
}
return false;
}
}
exports.HashStreamValidator = HashStreamValidator;
_HashStreamValidator_crc32cHash = new WeakMap(), _HashStreamValidator_md5Hash = new WeakMap(), _HashStreamValidator_md5Digest = new WeakMap();

View File

@@ -0,0 +1,93 @@
import { ServiceObject, MetadataCallback, SetMetadataResponse } from './nodejs-common/index.js';
import { BaseMetadata, SetMetadataOptions } from './nodejs-common/service-object.js';
import { Storage } from './storage.js';
export interface HmacKeyOptions {
projectId?: string;
}
export interface HmacKeyMetadata extends BaseMetadata {
accessId?: string;
etag?: string;
projectId?: string;
serviceAccountEmail?: string;
state?: string;
timeCreated?: string;
updated?: string;
}
export interface SetHmacKeyMetadataOptions {
/**
* This parameter is currently ignored.
*/
userProject?: string;
}
export interface SetHmacKeyMetadata {
state?: 'ACTIVE' | 'INACTIVE';
etag?: string;
}
export interface HmacKeyMetadataCallback {
(err: Error | null, metadata?: HmacKeyMetadata, apiResponse?: unknown): void;
}
export type HmacKeyMetadataResponse = [HmacKeyMetadata, unknown];
/**
* The API-formatted resource description of the HMAC key.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name HmacKey#metadata
* @type {object}
*/
/**
* An HmacKey object contains metadata of an HMAC key created from a
* service account through the {@link Storage} client using
* {@link Storage#createHmacKey}.
*
* See {@link https://cloud.google.com/storage/docs/authentication/hmackeys| HMAC keys documentation}
*
* @class
*/
export declare class HmacKey extends ServiceObject<HmacKey, HmacKeyMetadata> {
/**
* A reference to the {@link Storage} associated with this {@link HmacKey}
* instance.
* @name HmacKey#storage
* @type {Storage}
*/
storage: Storage;
private instanceRetryValue?;
/**
* @typedef {object} HmacKeyOptions
* @property {string} [projectId] The project ID of the project that owns
* the service account of the requested HMAC key. If not provided,
* the project ID used to instantiate the Storage client will be used.
*/
/**
* Constructs an HmacKey object.
*
* Note: this only create a local reference to an HMAC key, to create
* an HMAC key, use {@link Storage#createHmacKey}.
*
* @param {Storage} storage The Storage instance this HMAC key is
* attached to.
* @param {string} accessId The unique accessId for this HMAC key.
* @param {HmacKeyOptions} options Constructor configurations.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const hmacKey = storage.hmacKey('access-id');
* ```
*/
constructor(storage: Storage, accessId: string, options?: HmacKeyOptions);
/**
* Set the metadata for this object.
*
* @param {object} metadata - The metadata to set on this object.
* @param {object=} options - Configuration options.
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
setMetadata(metadata: HmacKeyMetadata, options?: SetMetadataOptions): Promise<SetMetadataResponse<HmacKeyMetadata>>;
setMetadata(metadata: HmacKeyMetadata, callback: MetadataCallback<HmacKeyMetadata>): void;
setMetadata(metadata: HmacKeyMetadata, options: SetMetadataOptions, callback: MetadataCallback<HmacKeyMetadata>): void;
}

View File

@@ -0,0 +1,336 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Object.defineProperty(exports, "__esModule", { value: true });
exports.HmacKey = void 0;
const index_js_1 = require("./nodejs-common/index.js");
const storage_js_1 = require("./storage.js");
const promisify_1 = require("@google-cloud/promisify");
/**
* The API-formatted resource description of the HMAC key.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name HmacKey#metadata
* @type {object}
*/
/**
* An HmacKey object contains metadata of an HMAC key created from a
* service account through the {@link Storage} client using
* {@link Storage#createHmacKey}.
*
* See {@link https://cloud.google.com/storage/docs/authentication/hmackeys| HMAC keys documentation}
*
* @class
*/
class HmacKey extends index_js_1.ServiceObject {
/**
* @typedef {object} HmacKeyOptions
* @property {string} [projectId] The project ID of the project that owns
* the service account of the requested HMAC key. If not provided,
* the project ID used to instantiate the Storage client will be used.
*/
/**
* Constructs an HmacKey object.
*
* Note: this only create a local reference to an HMAC key, to create
* an HMAC key, use {@link Storage#createHmacKey}.
*
* @param {Storage} storage The Storage instance this HMAC key is
* attached to.
* @param {string} accessId The unique accessId for this HMAC key.
* @param {HmacKeyOptions} options Constructor configurations.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const hmacKey = storage.hmacKey('access-id');
* ```
*/
constructor(storage, accessId, options) {
const methods = {
/**
* @typedef {object} DeleteHmacKeyOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* @typedef {array} DeleteHmacKeyResponse
* @property {object} 0 The full API response.
*/
/**
* @callback DeleteHmacKeyCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
/**
* Deletes an HMAC key.
* Key state must be set to `INACTIVE` prior to deletion.
* Caution: HMAC keys cannot be recovered once you delete them.
*
* The authenticated user must have `storage.hmacKeys.delete` permission for the project in which the key exists.
*
* @method HmacKey#delete
* @param {DeleteHmacKeyOptions} [options] Configuration options.
* @param {DeleteHmacKeyCallback} [callback] Callback function.
* @returns {Promise<DeleteHmacKeyResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Delete HMAC key after making the key inactive.
* //-
* const hmacKey = storage.hmacKey('ACCESS_ID');
* hmacKey.setMetadata({state: 'INACTIVE'}, (err, hmacKeyMetadata) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* hmacKey.delete((err) => {
* if (err) {
* console.error(err);
* return;
* }
* // The HMAC key is deleted.
* });
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* const hmacKey = storage.hmacKey('ACCESS_ID');
* hmacKey
* .setMetadata({state: 'INACTIVE'})
* .then(() => {
* return hmacKey.delete();
* });
* ```
*/
delete: true,
/**
* @callback GetHmacKeyCallback
* @param {?Error} err Request error, if any.
* @param {HmacKey} hmacKey this {@link HmacKey} instance.
* @param {object} apiResponse The full API response.
*/
/**
* @typedef {array} GetHmacKeyResponse
* @property {HmacKey} 0 This {@link HmacKey} instance.
* @property {object} 1 The full API response.
*/
/**
* @typedef {object} GetHmacKeyOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* Retrieves and populate an HMAC key's metadata, and return
* this {@link HmacKey} instance.
*
* HmacKey.get() does not give the HMAC key secret, as
* it is only returned on creation.
*
* The authenticated user must have `storage.hmacKeys.get` permission
* for the project in which the key exists.
*
* @method HmacKey#get
* @param {GetHmacKeyOptions} [options] Configuration options.
* @param {GetHmacKeyCallback} [callback] Callback function.
* @returns {Promise<GetHmacKeyResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Get the HmacKey's Metadata.
* //-
* storage.hmacKey('ACCESS_ID')
* .get((err, hmacKey) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* // do something with the returned HmacKey object.
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* storage.hmacKey('ACCESS_ID')
* .get()
* .then((data) => {
* const hmacKey = data[0];
* });
* ```
*/
get: true,
/**
* @typedef {object} GetHmacKeyMetadataOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* Retrieves and populate an HMAC key's metadata, and return
* the HMAC key's metadata as an object.
*
* HmacKey.getMetadata() does not give the HMAC key secret, as
* it is only returned on creation.
*
* The authenticated user must have `storage.hmacKeys.get` permission
* for the project in which the key exists.
*
* @method HmacKey#getMetadata
* @param {GetHmacKeyMetadataOptions} [options] Configuration options.
* @param {HmacKeyMetadataCallback} [callback] Callback function.
* @returns {Promise<HmacKeyMetadataResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Get the HmacKey's metadata and populate to the metadata property.
* //-
* storage.hmacKey('ACCESS_ID')
* .getMetadata((err, hmacKeyMetadata) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* console.log(hmacKeyMetadata);
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* storage.hmacKey('ACCESS_ID')
* .getMetadata()
* .then((data) => {
* const hmacKeyMetadata = data[0];
* console.log(hmacKeyMetadata);
* });
* ```
*/
getMetadata: true,
/**
* @typedef {object} SetHmacKeyMetadata Subset of {@link HmacKeyMetadata} to update.
* @property {string} state New state of the HmacKey. Either 'ACTIVE' or 'INACTIVE'.
* @property {string} [etag] Include an etag from a previous get HMAC key request
* to perform safe read-modify-write.
*/
/**
* @typedef {object} SetHmacKeyMetadataOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* @callback HmacKeyMetadataCallback
* @param {?Error} err Request error, if any.
* @param {HmacKeyMetadata} metadata The updated {@link HmacKeyMetadata} object.
* @param {object} apiResponse The full API response.
*/
/**
* @typedef {array} HmacKeyMetadataResponse
* @property {HmacKeyMetadata} 0 The updated {@link HmacKeyMetadata} object.
* @property {object} 1 The full API response.
*/
/**
* Updates the state of an HMAC key. See {@link SetHmacKeyMetadata} for
* valid states.
*
* @method HmacKey#setMetadata
* @param {SetHmacKeyMetadata} metadata The new metadata.
* @param {SetHmacKeyMetadataOptions} [options] Configuration options.
* @param {HmacKeyMetadataCallback} [callback] Callback function.
* @returns {Promise<HmacKeyMetadataResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* const metadata = {
* state: 'INACTIVE',
* };
*
* storage.hmacKey('ACCESS_ID')
* .setMetadata(metadata, (err, hmacKeyMetadata) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* console.log(hmacKeyMetadata);
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* storage.hmacKey('ACCESS_ID')
* .setMetadata(metadata)
* .then((data) => {
* const hmacKeyMetadata = data[0];
* console.log(hmacKeyMetadata);
* });
* ```
*/
setMetadata: {
reqOpts: {
method: 'PUT',
},
},
};
const projectId = (options && options.projectId) || storage.projectId;
super({
parent: storage,
id: accessId,
baseUrl: `/projects/${projectId}/hmacKeys`,
methods,
});
this.storage = storage;
this.instanceRetryValue = storage.retryOptions.autoRetry;
}
setMetadata(metadata, optionsOrCallback, cb) {
// ETag preconditions are not currently supported. Retries should be disabled if the idempotency strategy is not set to RetryAlways
if (this.storage.retryOptions.idempotencyStrategy !==
storage_js_1.IdempotencyStrategy.RetryAlways) {
this.storage.retryOptions.autoRetry = false;
}
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : {};
cb =
typeof optionsOrCallback === 'function'
? optionsOrCallback
: cb;
super
.setMetadata(metadata, options)
.then(resp => cb(null, ...resp))
.catch(cb)
.finally(() => {
this.storage.retryOptions.autoRetry = this.instanceRetryValue;
});
}
}
exports.HmacKey = HmacKey;
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
(0, promisify_1.promisifyAll)(HmacKey);

View File

@@ -0,0 +1,117 @@
import { Bucket } from './bucket.js';
export interface GetPolicyOptions {
userProject?: string;
requestedPolicyVersion?: number;
}
export type GetPolicyResponse = [Policy, unknown];
/**
* @callback GetPolicyCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The policy.
* @param {object} apiResponse The full API response.
*/
export interface GetPolicyCallback {
(err?: Error | null, acl?: Policy, apiResponse?: unknown): void;
}
/**
* @typedef {object} SetPolicyOptions
* @param {string} [userProject] The ID of the project which will be
* billed for the request.
*/
export interface SetPolicyOptions {
userProject?: string;
}
/**
* @typedef {array} SetPolicyResponse
* @property {object} 0 The policy.
* @property {object} 1 The full API response.
*/
export type SetPolicyResponse = [Policy, unknown];
/**
* @callback SetPolicyCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The policy.
* @param {object} apiResponse The full API response.
*/
export interface SetPolicyCallback {
(err?: Error | null, acl?: Policy, apiResponse?: object): void;
}
export interface Policy {
bindings: PolicyBinding[];
version?: number;
etag?: string;
}
export interface PolicyBinding {
role: string;
members: string[];
condition?: Expr;
}
export interface Expr {
title?: string;
description?: string;
expression: string;
}
/**
* @typedef {array} TestIamPermissionsResponse
* @property {object} 0 A subset of permissions that the caller is allowed.
* @property {object} 1 The full API response.
*/
export type TestIamPermissionsResponse = [{
[key: string]: boolean;
}, unknown];
/**
* @callback TestIamPermissionsCallback
* @param {?Error} err Request error, if any.
* @param {object} acl A subset of permissions that the caller is allowed.
* @param {object} apiResponse The full API response.
*/
export interface TestIamPermissionsCallback {
(err?: Error | null, acl?: {
[key: string]: boolean;
} | null, apiResponse?: unknown): void;
}
/**
* @typedef {object} TestIamPermissionsOptions Configuration options for Iam#testPermissions().
* @param {string} [userProject] The ID of the project which will be
* billed for the request.
*/
export interface TestIamPermissionsOptions {
userProject?: string;
}
export declare enum IAMExceptionMessages {
POLICY_OBJECT_REQUIRED = "A policy object is required.",
PERMISSIONS_REQUIRED = "Permissions are required."
}
/**
* Get and set IAM policies for your Cloud Storage bucket.
*
* See {@link https://cloud.google.com/storage/docs/access-control/iam#short_title_iam_management| Cloud Storage IAM Management}
* See {@link https://cloud.google.com/iam/docs/granting-changing-revoking-access| Granting, Changing, and Revoking Access}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @constructor Iam
*
* @param {Bucket} bucket The parent instance.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* // bucket.iam
* ```
*/
declare class Iam {
private request_;
private resourceId_;
constructor(bucket: Bucket);
getPolicy(options?: GetPolicyOptions): Promise<GetPolicyResponse>;
getPolicy(options: GetPolicyOptions, callback: GetPolicyCallback): void;
getPolicy(callback: GetPolicyCallback): void;
setPolicy(policy: Policy, options?: SetPolicyOptions): Promise<SetPolicyResponse>;
setPolicy(policy: Policy, callback: SetPolicyCallback): void;
setPolicy(policy: Policy, options: SetPolicyOptions, callback: SetPolicyCallback): void;
testPermissions(permissions: string | string[], options?: TestIamPermissionsOptions): Promise<TestIamPermissionsResponse>;
testPermissions(permissions: string | string[], callback: TestIamPermissionsCallback): void;
testPermissions(permissions: string | string[], options: TestIamPermissionsOptions, callback: TestIamPermissionsCallback): void;
}
export { Iam };

View File

@@ -0,0 +1,306 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Object.defineProperty(exports, "__esModule", { value: true });
exports.Iam = exports.IAMExceptionMessages = void 0;
const promisify_1 = require("@google-cloud/promisify");
const util_js_1 = require("./util.js");
var IAMExceptionMessages;
(function (IAMExceptionMessages) {
IAMExceptionMessages["POLICY_OBJECT_REQUIRED"] = "A policy object is required.";
IAMExceptionMessages["PERMISSIONS_REQUIRED"] = "Permissions are required.";
})(IAMExceptionMessages || (exports.IAMExceptionMessages = IAMExceptionMessages = {}));
/**
* Get and set IAM policies for your Cloud Storage bucket.
*
* See {@link https://cloud.google.com/storage/docs/access-control/iam#short_title_iam_management| Cloud Storage IAM Management}
* See {@link https://cloud.google.com/iam/docs/granting-changing-revoking-access| Granting, Changing, and Revoking Access}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @constructor Iam
*
* @param {Bucket} bucket The parent instance.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* // bucket.iam
* ```
*/
class Iam {
constructor(bucket) {
this.request_ = bucket.request.bind(bucket);
this.resourceId_ = 'buckets/' + bucket.getId();
}
/**
* @typedef {object} GetPolicyOptions Requested options for IAM#getPolicy().
* @property {number} [requestedPolicyVersion] The version of IAM policies to
* request. If a policy with a condition is requested without setting
* this, the server will return an error. This must be set to a value
* of 3 to retrieve IAM policies containing conditions. This is to
* prevent client code that isn't aware of IAM conditions from
* interpreting and modifying policies incorrectly. The service might
* return a policy with version lower than the one that was requested,
* based on the feature syntax in the policy fetched.
* See {@link https://cloud.google.com/iam/docs/policies#versions| IAM Policy versions}
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
*/
/**
* @typedef {array} GetPolicyResponse
* @property {Policy} 0 The policy.
* @property {object} 1 The full API response.
*/
/**
* @typedef {object} Policy
* @property {PolicyBinding[]} policy.bindings Bindings associate members with roles.
* @property {string} [policy.etag] Etags are used to perform a read-modify-write.
* @property {number} [policy.version] The syntax schema version of the Policy.
* To set an IAM policy with conditional binding, this field must be set to
* 3 or greater.
* See {@link https://cloud.google.com/iam/docs/policies#versions| IAM Policy versions}
*/
/**
* @typedef {object} PolicyBinding
* @property {string} role Role that is assigned to members.
* @property {string[]} members Specifies the identities requesting access for the bucket.
* @property {Expr} [condition] The condition that is associated with this binding.
*/
/**
* @typedef {object} Expr
* @property {string} [title] An optional title for the expression, i.e. a
* short string describing its purpose. This can be used e.g. in UIs
* which allow to enter the expression.
* @property {string} [description] An optional description of the
* expression. This is a longer text which describes the expression,
* e.g. when hovered over it in a UI.
* @property {string} expression Textual representation of an expression in
* Common Expression Language syntax. The application context of the
* containing message determines which well-known feature set of CEL
* is supported.The condition that is associated with this binding.
*
* @see [Condition] https://cloud.google.com/storage/docs/access-control/iam#conditions
*/
/**
* Get the IAM policy.
*
* @param {GetPolicyOptions} [options] Request options.
* @param {GetPolicyCallback} [callback] Callback function.
* @returns {Promise<GetPolicyResponse>}
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/getIamPolicy| Buckets: setIamPolicy API Documentation}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* bucket.iam.getPolicy(
* {requestedPolicyVersion: 3},
* function(err, policy, apiResponse) {
*
* },
* );
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.getPolicy({requestedPolicyVersion: 3})
* .then(function(data) {
* const policy = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_view_bucket_iam_members
* Example of retrieving a bucket's IAM policy:
*/
getPolicy(optionsOrCallback, callback) {
const { options, callback: cb } = (0, util_js_1.normalize)(optionsOrCallback, callback);
const qs = {};
if (options.userProject) {
qs.userProject = options.userProject;
}
if (options.requestedPolicyVersion !== null &&
options.requestedPolicyVersion !== undefined) {
qs.optionsRequestedPolicyVersion = options.requestedPolicyVersion;
}
this.request_({
uri: '/iam',
qs,
}, cb);
}
/**
* Set the IAM policy.
*
* @throws {Error} If no policy is provided.
*
* @param {Policy} policy The policy.
* @param {SetPolicyOptions} [options] Configuration options.
* @param {SetPolicyCallback} callback Callback function.
* @returns {Promise<SetPolicyResponse>}
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/setIamPolicy| Buckets: setIamPolicy API Documentation}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* const myPolicy = {
* bindings: [
* {
* role: 'roles/storage.admin',
* members:
* ['serviceAccount:myotherproject@appspot.gserviceaccount.com']
* }
* ]
* };
*
* bucket.iam.setPolicy(myPolicy, function(err, policy, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.setPolicy(myPolicy).then(function(data) {
* const policy = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_add_bucket_iam_member
* Example of adding to a bucket's IAM policy:
*
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_remove_bucket_iam_member
* Example of removing from a bucket's IAM policy:
*/
setPolicy(policy, optionsOrCallback, callback) {
if (policy === null || typeof policy !== 'object') {
throw new Error(IAMExceptionMessages.POLICY_OBJECT_REQUIRED);
}
const { options, callback: cb } = (0, util_js_1.normalize)(optionsOrCallback, callback);
let maxRetries;
if (policy.etag === undefined) {
maxRetries = 0;
}
this.request_({
method: 'PUT',
uri: '/iam',
maxRetries,
json: Object.assign({
resourceId: this.resourceId_,
}, policy),
qs: options,
}, cb);
}
/**
* Test a set of permissions for a resource.
*
* @throws {Error} If permissions are not provided.
*
* @param {string|string[]} permissions The permission(s) to test for.
* @param {TestIamPermissionsOptions} [options] Configuration object.
* @param {TestIamPermissionsCallback} [callback] Callback function.
* @returns {Promise<TestIamPermissionsResponse>}
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/testIamPermissions| Buckets: testIamPermissions API Documentation}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* //-
* // Test a single permission.
* //-
* const test = 'storage.buckets.delete';
*
* bucket.iam.testPermissions(test, function(err, permissions, apiResponse) {
* console.log(permissions);
* // {
* // "storage.buckets.delete": true
* // }
* });
*
* //-
* // Test several permissions at once.
* //-
* const tests = [
* 'storage.buckets.delete',
* 'storage.buckets.get'
* ];
*
* bucket.iam.testPermissions(tests, function(err, permissions) {
* console.log(permissions);
* // {
* // "storage.buckets.delete": false,
* // "storage.buckets.get": true
* // }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.testPermissions(test).then(function(data) {
* const permissions = data[0];
* const apiResponse = data[1];
* });
* ```
*/
testPermissions(permissions, optionsOrCallback, callback) {
if (!Array.isArray(permissions) && typeof permissions !== 'string') {
throw new Error(IAMExceptionMessages.PERMISSIONS_REQUIRED);
}
const { options, callback: cb } = (0, util_js_1.normalize)(optionsOrCallback, callback);
const permissionsArray = Array.isArray(permissions)
? permissions
: [permissions];
const req = Object.assign({
permissions: permissionsArray,
}, options);
this.request_({
uri: '/iam/testPermissions',
qs: req,
useQuerystring: true,
}, (err, resp) => {
if (err) {
cb(err, null, resp);
return;
}
const availablePermissions = Array.isArray(resp.permissions)
? resp.permissions
: [];
const permissionsHash = permissionsArray.reduce((acc, permission) => {
acc[permission] = availablePermissions.indexOf(permission) > -1;
return acc;
}, {});
cb(null, permissionsHash, resp);
});
}
}
exports.Iam = Iam;
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
(0, promisify_1.promisifyAll)(Iam);

View File

@@ -0,0 +1,57 @@
/**
* The `@google-cloud/storage` package has a single named export which is the
* {@link Storage} (ES6) class, which should be instantiated with `new`.
*
* See {@link Storage} and {@link ClientConfig} for client methods and
* configuration options.
*
* @module {Storage} @google-cloud/storage
* @alias nodejs-storage
*
* @example
* Install the client library with <a href="https://www.npmjs.com/">npm</a>:
* ```
* npm install --save @google-cloud/storage
* ```
*
* @example
* Import the client library
* ```
* const {Storage} = require('@google-cloud/storage');
* ```
*
* @example
* Create a client that uses <a
* href="https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application">Application
* Default Credentials (ADC)</a>:
* ```
* const storage = new Storage();
* ```
*
* @example
* Create a client with <a
* href="https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually">explicit
* credentials</a>:
* ```
* const storage = new Storage({ projectId:
* 'your-project-id', keyFilename: '/path/to/keyfile.json'
* });
* ```
*
* @example <caption>include:samples/quickstart.js</caption>
* region_tag:storage_quickstart
* Full quickstart example:
*/
export { ApiError } from './nodejs-common/index.js';
export { BucketCallback, BucketOptions, CreateBucketQuery, CreateBucketRequest, CreateBucketResponse, CreateHmacKeyCallback, CreateHmacKeyOptions, CreateHmacKeyResponse, GetBucketsCallback, GetBucketsRequest, GetBucketsResponse, GetHmacKeysCallback, GetHmacKeysOptions, GetHmacKeysResponse, GetServiceAccountCallback, GetServiceAccountOptions, GetServiceAccountResponse, HmacKeyResourceResponse, IdempotencyStrategy, PreconditionOptions, RETRYABLE_ERR_FN_DEFAULT, ServiceAccount, Storage, StorageOptions, } from './storage.js';
export { AclMetadata, AccessControlObject, AclOptions, AddAclCallback, AddAclOptions, AddAclResponse, GetAclCallback, GetAclOptions, GetAclResponse, RemoveAclCallback, RemoveAclOptions, RemoveAclResponse, UpdateAclCallback, UpdateAclOptions, UpdateAclResponse, } from './acl.js';
export { Bucket, BucketExistsCallback, BucketExistsOptions, BucketExistsResponse, BucketLockCallback, BucketLockResponse, BucketMetadata, CombineCallback, CombineOptions, CombineResponse, CreateChannelCallback, CreateChannelConfig, CreateChannelOptions, CreateChannelResponse, CreateNotificationCallback, CreateNotificationOptions, CreateNotificationResponse, DeleteBucketCallback, DeleteBucketOptions, DeleteBucketResponse, DeleteFilesCallback, DeleteFilesOptions, DeleteLabelsCallback, DeleteLabelsResponse, DisableRequesterPaysCallback, DisableRequesterPaysResponse, EnableRequesterPaysCallback, EnableRequesterPaysResponse, GetBucketCallback, GetBucketMetadataCallback, GetBucketMetadataOptions, GetBucketMetadataResponse, GetBucketOptions, GetBucketResponse, GetBucketSignedUrlConfig, GetFilesCallback, GetFilesOptions, GetFilesResponse, GetLabelsCallback, GetLabelsOptions, GetLabelsResponse, GetNotificationsCallback, GetNotificationsOptions, GetNotificationsResponse, Labels, LifecycleAction, LifecycleCondition, LifecycleRule, MakeBucketPrivateCallback, MakeBucketPrivateOptions, MakeBucketPrivateResponse, MakeBucketPublicCallback, MakeBucketPublicOptions, MakeBucketPublicResponse, SetBucketMetadataCallback, SetBucketMetadataOptions, SetBucketMetadataResponse, SetBucketStorageClassCallback, SetBucketStorageClassOptions, SetLabelsCallback, SetLabelsOptions, SetLabelsResponse, UploadCallback, UploadOptions, UploadResponse, } from './bucket.js';
export * from './crc32c.js';
export { Channel, StopCallback } from './channel.js';
export { CopyCallback, CopyOptions, CopyResponse, CreateReadStreamOptions, CreateResumableUploadCallback, CreateResumableUploadOptions, CreateResumableUploadResponse, CreateWriteStreamOptions, DeleteFileCallback, DeleteFileOptions, DeleteFileResponse, DownloadCallback, DownloadOptions, DownloadResponse, EncryptionKeyOptions, File, FileExistsCallback, FileExistsOptions, FileExistsResponse, FileMetadata, FileOptions, GetExpirationDateCallback, GetExpirationDateResponse, GetFileCallback, GetFileMetadataCallback, GetFileMetadataOptions, GetFileMetadataResponse, GetFileOptions, GetFileResponse, GenerateSignedPostPolicyV2Callback, GenerateSignedPostPolicyV2Options, GenerateSignedPostPolicyV2Response, GenerateSignedPostPolicyV4Callback, GenerateSignedPostPolicyV4Options, GenerateSignedPostPolicyV4Response, GetSignedUrlConfig, MakeFilePrivateCallback, MakeFilePrivateOptions, MakeFilePrivateResponse, MakeFilePublicCallback, MakeFilePublicResponse, MoveCallback, MoveOptions, MoveResponse, MoveFileAtomicOptions, MoveFileAtomicCallback, MoveFileAtomicResponse, PolicyDocument, PolicyFields, PredefinedAcl, RotateEncryptionKeyCallback, RotateEncryptionKeyOptions, RotateEncryptionKeyResponse, SaveCallback, SaveData, SaveOptions, SetFileMetadataCallback, SetFileMetadataOptions, SetFileMetadataResponse, SetStorageClassCallback, SetStorageClassOptions, SetStorageClassResponse, SignedPostPolicyV4Output, } from './file.js';
export * from './hash-stream-validator.js';
export { HmacKey, HmacKeyMetadata, HmacKeyMetadataCallback, HmacKeyMetadataResponse, SetHmacKeyMetadata, SetHmacKeyMetadataOptions, } from './hmacKey.js';
export { GetPolicyCallback, GetPolicyOptions, GetPolicyResponse, Iam, Policy, SetPolicyCallback, SetPolicyOptions, SetPolicyResponse, TestIamPermissionsCallback, TestIamPermissionsOptions, TestIamPermissionsResponse, } from './iam.js';
export { DeleteNotificationCallback, DeleteNotificationOptions, GetNotificationCallback, GetNotificationMetadataCallback, GetNotificationMetadataOptions, GetNotificationMetadataResponse, GetNotificationOptions, GetNotificationResponse, Notification, NotificationMetadata, } from './notification.js';
export { GetSignedUrlCallback, GetSignedUrlResponse } from './signer.js';
export * from './transfer-manager.js';

View File

@@ -0,0 +1,95 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __exportStar = (this && this.__exportStar) || function(m, exports) {
for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.Notification = exports.Iam = exports.HmacKey = exports.File = exports.Channel = exports.Bucket = exports.Storage = exports.RETRYABLE_ERR_FN_DEFAULT = exports.IdempotencyStrategy = exports.ApiError = void 0;
/**
* The `@google-cloud/storage` package has a single named export which is the
* {@link Storage} (ES6) class, which should be instantiated with `new`.
*
* See {@link Storage} and {@link ClientConfig} for client methods and
* configuration options.
*
* @module {Storage} @google-cloud/storage
* @alias nodejs-storage
*
* @example
* Install the client library with <a href="https://www.npmjs.com/">npm</a>:
* ```
* npm install --save @google-cloud/storage
* ```
*
* @example
* Import the client library
* ```
* const {Storage} = require('@google-cloud/storage');
* ```
*
* @example
* Create a client that uses <a
* href="https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application">Application
* Default Credentials (ADC)</a>:
* ```
* const storage = new Storage();
* ```
*
* @example
* Create a client with <a
* href="https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually">explicit
* credentials</a>:
* ```
* const storage = new Storage({ projectId:
* 'your-project-id', keyFilename: '/path/to/keyfile.json'
* });
* ```
*
* @example <caption>include:samples/quickstart.js</caption>
* region_tag:storage_quickstart
* Full quickstart example:
*/
var index_js_1 = require("./nodejs-common/index.js");
Object.defineProperty(exports, "ApiError", { enumerable: true, get: function () { return index_js_1.ApiError; } });
var storage_js_1 = require("./storage.js");
Object.defineProperty(exports, "IdempotencyStrategy", { enumerable: true, get: function () { return storage_js_1.IdempotencyStrategy; } });
Object.defineProperty(exports, "RETRYABLE_ERR_FN_DEFAULT", { enumerable: true, get: function () { return storage_js_1.RETRYABLE_ERR_FN_DEFAULT; } });
Object.defineProperty(exports, "Storage", { enumerable: true, get: function () { return storage_js_1.Storage; } });
var bucket_js_1 = require("./bucket.js");
Object.defineProperty(exports, "Bucket", { enumerable: true, get: function () { return bucket_js_1.Bucket; } });
__exportStar(require("./crc32c.js"), exports);
var channel_js_1 = require("./channel.js");
Object.defineProperty(exports, "Channel", { enumerable: true, get: function () { return channel_js_1.Channel; } });
var file_js_1 = require("./file.js");
Object.defineProperty(exports, "File", { enumerable: true, get: function () { return file_js_1.File; } });
__exportStar(require("./hash-stream-validator.js"), exports);
var hmacKey_js_1 = require("./hmacKey.js");
Object.defineProperty(exports, "HmacKey", { enumerable: true, get: function () { return hmacKey_js_1.HmacKey; } });
var iam_js_1 = require("./iam.js");
Object.defineProperty(exports, "Iam", { enumerable: true, get: function () { return iam_js_1.Iam; } });
var notification_js_1 = require("./notification.js");
Object.defineProperty(exports, "Notification", { enumerable: true, get: function () { return notification_js_1.Notification; } });
__exportStar(require("./transfer-manager.js"), exports);

View File

@@ -0,0 +1,19 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
export { GoogleAuthOptions } from 'google-auth-library';
export { Service, ServiceConfig, ServiceOptions, StreamRequestOptions, } from './service.js';
export { BaseMetadata, DeleteCallback, ExistsCallback, GetConfig, InstanceResponseCallback, Interceptor, MetadataCallback, MetadataResponse, Methods, ResponseCallback, ServiceObject, ServiceObjectConfig, ServiceObjectParent, SetMetadataResponse, } from './service-object.js';
export { Abortable, AbortableDuplex, ApiError, BodyResponseCallback, DecorateRequestOptions, ResponseBody, util, } from './util.js';

View File

@@ -0,0 +1,10 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.util = exports.ApiError = exports.ServiceObject = exports.Service = void 0;
var service_js_1 = require("./service.js");
Object.defineProperty(exports, "Service", { enumerable: true, get: function () { return service_js_1.Service; } });
var service_object_js_1 = require("./service-object.js");
Object.defineProperty(exports, "ServiceObject", { enumerable: true, get: function () { return service_object_js_1.ServiceObject; } });
var util_js_1 = require("./util.js");
Object.defineProperty(exports, "ApiError", { enumerable: true, get: function () { return util_js_1.ApiError; } });
Object.defineProperty(exports, "util", { enumerable: true, get: function () { return util_js_1.util; } });

View File

@@ -0,0 +1,217 @@
import { EventEmitter } from 'events';
import * as r from 'teeny-request';
import { ApiError, BodyResponseCallback, DecorateRequestOptions } from './util.js';
export type RequestResponse = [unknown, r.Response];
export interface ServiceObjectParent {
interceptors: Interceptor[];
getRequestInterceptors(): Function[];
requestStream(reqOpts: DecorateRequestOptions): r.Request;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
}
export interface Interceptor {
request(opts: r.Options): DecorateRequestOptions;
}
export type GetMetadataOptions = object;
export type MetadataResponse<K> = [K, r.Response];
export type MetadataCallback<K> = (err: Error | null, metadata?: K, apiResponse?: r.Response) => void;
export type ExistsOptions = object;
export interface ExistsCallback {
(err: Error | null, exists?: boolean): void;
}
export interface ServiceObjectConfig {
/**
* The base URL to make API requests to.
*/
baseUrl?: string;
/**
* The method which creates this object.
*/
createMethod?: Function;
/**
* The identifier of the object. For example, the name of a Storage bucket or
* Pub/Sub topic.
*/
id?: string;
/**
* A map of each method name that should be inherited.
*/
methods?: Methods;
/**
* The parent service instance. For example, an instance of Storage if the
* object is Bucket.
*/
parent: ServiceObjectParent;
/**
* Override of projectId, used to allow access to resources in another project.
* For example, a BigQuery dataset in another project to which the user has been
* granted permission.
*/
projectId?: string;
}
export interface Methods {
[methodName: string]: {
reqOpts?: r.CoreOptions;
} | boolean;
}
export interface InstanceResponseCallback<T> {
(err: ApiError | null, instance?: T | null, apiResponse?: r.Response): void;
}
export interface CreateOptions {
}
export type CreateResponse<T> = any[];
export interface CreateCallback<T> {
(err: ApiError | null, instance?: T | null, ...args: any[]): void;
}
export type DeleteOptions = {
ignoreNotFound?: boolean;
ifGenerationMatch?: number | string;
ifGenerationNotMatch?: number | string;
ifMetagenerationMatch?: number | string;
ifMetagenerationNotMatch?: number | string;
} & object;
export interface DeleteCallback {
(err: Error | null, apiResponse?: r.Response): void;
}
export interface GetConfig {
/**
* Create the object if it doesn't already exist.
*/
autoCreate?: boolean;
}
export type GetOrCreateOptions = GetConfig & CreateOptions;
export type GetResponse<T> = [T, r.Response];
export interface ResponseCallback {
(err?: Error | null, apiResponse?: r.Response): void;
}
export type SetMetadataResponse<K> = [K];
export type SetMetadataOptions = object;
export interface BaseMetadata {
id?: string;
kind?: string;
etag?: string;
selfLink?: string;
[key: string]: unknown;
}
/**
* ServiceObject is a base class, meant to be inherited from by a "service
* object," like a BigQuery dataset or Storage bucket.
*
* Most of the time, these objects share common functionality; they can be
* created or deleted, and you can get or set their metadata.
*
* By inheriting from this class, a service object will be extended with these
* shared behaviors. Note that any method can be overridden when the service
* object requires specific behavior.
*/
declare class ServiceObject<T, K extends BaseMetadata> extends EventEmitter {
metadata: K;
baseUrl?: string;
parent: ServiceObjectParent;
id?: string;
private createMethod?;
protected methods: Methods;
interceptors: Interceptor[];
projectId?: string;
constructor(config: ServiceObjectConfig);
/**
* Create the object.
*
* @param {object=} options - Configuration object.
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.instance - The instance.
* @param {object} callback.apiResponse - The full API response.
*/
create(options?: CreateOptions): Promise<CreateResponse<T>>;
create(options: CreateOptions, callback: CreateCallback<T>): void;
create(callback: CreateCallback<T>): void;
/**
* Delete the object.
*
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
delete(options?: DeleteOptions): Promise<[r.Response]>;
delete(options: DeleteOptions, callback: DeleteCallback): void;
delete(callback: DeleteCallback): void;
/**
* Check if the object exists.
*
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {boolean} callback.exists - Whether the object exists or not.
*/
exists(options?: ExistsOptions): Promise<[boolean]>;
exists(options: ExistsOptions, callback: ExistsCallback): void;
exists(callback: ExistsCallback): void;
/**
* Get the object if it exists. Optionally have the object created if an
* options object is provided with `autoCreate: true`.
*
* @param {object=} options - The configuration object that will be used to
* create the object if necessary.
* @param {boolean} options.autoCreate - Create the object if it doesn't already exist.
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.instance - The instance.
* @param {object} callback.apiResponse - The full API response.
*/
get(options?: GetOrCreateOptions): Promise<GetResponse<T>>;
get(callback: InstanceResponseCallback<T>): void;
get(options: GetOrCreateOptions, callback: InstanceResponseCallback<T>): void;
/**
* Get the metadata of this object.
*
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.metadata - The metadata for this object.
* @param {object} callback.apiResponse - The full API response.
*/
getMetadata(options?: GetMetadataOptions): Promise<MetadataResponse<K>>;
getMetadata(options: GetMetadataOptions, callback: MetadataCallback<K>): void;
getMetadata(callback: MetadataCallback<K>): void;
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors(): Function[];
/**
* Set the metadata for this object.
*
* @param {object} metadata - The metadata to set on this object.
* @param {object=} options - Configuration options.
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
setMetadata(metadata: K, options?: SetMetadataOptions): Promise<SetMetadataResponse<K>>;
setMetadata(metadata: K, callback: MetadataCallback<K>): void;
setMetadata(metadata: K, options: SetMetadataOptions, callback: MetadataCallback<K>): void;
/**
* Make an authenticated API request.
*
* @private
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
private request_;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
request(reqOpts: DecorateRequestOptions): Promise<RequestResponse>;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts: DecorateRequestOptions): r.Request;
}
export { ServiceObject };

View File

@@ -0,0 +1,292 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.ServiceObject = void 0;
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
const promisify_1 = require("@google-cloud/promisify");
const events_1 = require("events");
const util_js_1 = require("./util.js");
/**
* ServiceObject is a base class, meant to be inherited from by a "service
* object," like a BigQuery dataset or Storage bucket.
*
* Most of the time, these objects share common functionality; they can be
* created or deleted, and you can get or set their metadata.
*
* By inheriting from this class, a service object will be extended with these
* shared behaviors. Note that any method can be overridden when the service
* object requires specific behavior.
*/
// eslint-disable-next-line @typescript-eslint/no-explicit-any
class ServiceObject extends events_1.EventEmitter {
/*
* @constructor
* @alias module:common/service-object
*
* @private
*
* @param {object} config - Configuration object.
* @param {string} config.baseUrl - The base URL to make API requests to.
* @param {string} config.createMethod - The method which creates this object.
* @param {string=} config.id - The identifier of the object. For example, the
* name of a Storage bucket or Pub/Sub topic.
* @param {object=} config.methods - A map of each method name that should be inherited.
* @param {object} config.methods[].reqOpts - Default request options for this
* particular method. A common use case is when `setMetadata` requires a
* `PUT` method to override the default `PATCH`.
* @param {object} config.parent - The parent service instance. For example, an
* instance of Storage if the object is Bucket.
*/
constructor(config) {
super();
this.metadata = {};
this.baseUrl = config.baseUrl;
this.parent = config.parent; // Parent class.
this.id = config.id; // Name or ID (e.g. dataset ID, bucket name, etc).
this.createMethod = config.createMethod;
this.methods = config.methods || {};
this.interceptors = [];
this.projectId = config.projectId;
if (config.methods) {
// This filters the ServiceObject instance (e.g. a "File") to only have
// the configured methods. We make a couple of exceptions for core-
// functionality ("request()" and "getRequestInterceptors()")
Object.getOwnPropertyNames(ServiceObject.prototype)
.filter(methodName => {
return (
// All ServiceObjects need `request` and `getRequestInterceptors`.
// clang-format off
!/^request/.test(methodName) &&
!/^getRequestInterceptors/.test(methodName) &&
// clang-format on
// The ServiceObject didn't redefine the method.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this[methodName] ===
// eslint-disable-next-line @typescript-eslint/no-explicit-any
ServiceObject.prototype[methodName] &&
// This method isn't wanted.
!config.methods[methodName]);
})
.forEach(methodName => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this[methodName] = undefined;
});
}
}
create(optionsOrCallback, callback) {
// eslint-disable-next-line @typescript-eslint/no-this-alias
const self = this;
const args = [this.id];
if (typeof optionsOrCallback === 'function') {
callback = optionsOrCallback;
}
if (typeof optionsOrCallback === 'object') {
args.push(optionsOrCallback);
}
// Wrap the callback to return *this* instance of the object, not the
// newly-created one.
// tslint: disable-next-line no-any
function onCreate(...args) {
const [err, instance] = args;
if (!err) {
self.metadata = instance.metadata;
if (self.id && instance.metadata) {
self.id = instance.metadata.id;
}
args[1] = self; // replace the created `instance` with this one.
}
callback(...args);
}
args.push(onCreate);
// eslint-disable-next-line prefer-spread
this.createMethod.apply(null, args);
}
delete(optionsOrCallback, cb) {
var _a;
const [options, callback] = util_js_1.util.maybeOptionsOrCallback(optionsOrCallback, cb);
const ignoreNotFound = options.ignoreNotFound;
delete options.ignoreNotFound;
const methodConfig = (typeof this.methods.delete === 'object' && this.methods.delete) || {};
const reqOpts = {
method: 'DELETE',
uri: '',
...methodConfig.reqOpts,
qs: {
...(_a = methodConfig.reqOpts) === null || _a === void 0 ? void 0 : _a.qs,
...options,
},
};
// The `request` method may have been overridden to hold any special
// behavior. Ensure we call the original `request` method.
ServiceObject.prototype.request.call(this, reqOpts, (err, body, res) => {
if (err) {
if (err.code === 404 && ignoreNotFound) {
err = null;
}
}
callback(err, res);
});
}
exists(optionsOrCallback, cb) {
const [options, callback] = util_js_1.util.maybeOptionsOrCallback(optionsOrCallback, cb);
this.get(options, err => {
if (err) {
if (err.code === 404) {
callback(null, false);
}
else {
callback(err);
}
return;
}
callback(null, true);
});
}
get(optionsOrCallback, cb) {
// eslint-disable-next-line @typescript-eslint/no-this-alias
const self = this;
const [opts, callback] = util_js_1.util.maybeOptionsOrCallback(optionsOrCallback, cb);
const options = Object.assign({}, opts);
const autoCreate = options.autoCreate && typeof this.create === 'function';
delete options.autoCreate;
function onCreate(err, instance, apiResponse) {
if (err) {
if (err.code === 409) {
self.get(options, callback);
return;
}
callback(err, null, apiResponse);
return;
}
callback(null, instance, apiResponse);
}
this.getMetadata(options, (err, metadata) => {
if (err) {
if (err.code === 404 && autoCreate) {
const args = [];
if (Object.keys(options).length > 0) {
args.push(options);
}
args.push(onCreate);
self.create(...args);
return;
}
callback(err, null, metadata);
return;
}
callback(null, self, metadata);
});
}
getMetadata(optionsOrCallback, cb) {
var _a;
const [options, callback] = util_js_1.util.maybeOptionsOrCallback(optionsOrCallback, cb);
const methodConfig = (typeof this.methods.getMetadata === 'object' &&
this.methods.getMetadata) ||
{};
const reqOpts = {
uri: '',
...methodConfig.reqOpts,
qs: {
...(_a = methodConfig.reqOpts) === null || _a === void 0 ? void 0 : _a.qs,
...options,
},
};
// The `request` method may have been overridden to hold any special
// behavior. Ensure we call the original `request` method.
ServiceObject.prototype.request.call(this, reqOpts, (err, body, res) => {
this.metadata = body;
callback(err, this.metadata, res);
});
}
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors() {
// Interceptors should be returned in the order they were assigned.
const localInterceptors = this.interceptors
.filter(interceptor => typeof interceptor.request === 'function')
.map(interceptor => interceptor.request);
return this.parent.getRequestInterceptors().concat(localInterceptors);
}
setMetadata(metadata, optionsOrCallback, cb) {
var _a, _b;
const [options, callback] = util_js_1.util.maybeOptionsOrCallback(optionsOrCallback, cb);
const methodConfig = (typeof this.methods.setMetadata === 'object' &&
this.methods.setMetadata) ||
{};
const reqOpts = {
method: 'PATCH',
uri: '',
...methodConfig.reqOpts,
json: {
...(_a = methodConfig.reqOpts) === null || _a === void 0 ? void 0 : _a.json,
...metadata,
},
qs: {
...(_b = methodConfig.reqOpts) === null || _b === void 0 ? void 0 : _b.qs,
...options,
},
};
// The `request` method may have been overridden to hold any special
// behavior. Ensure we call the original `request` method.
ServiceObject.prototype.request.call(this, reqOpts, (err, body, res) => {
this.metadata = body;
callback(err, this.metadata, res);
});
}
request_(reqOpts, callback) {
reqOpts = { ...reqOpts };
if (this.projectId) {
reqOpts.projectId = this.projectId;
}
const isAbsoluteUrl = reqOpts.uri.indexOf('http') === 0;
const uriComponents = [this.baseUrl, this.id || '', reqOpts.uri];
if (isAbsoluteUrl) {
uriComponents.splice(0, uriComponents.indexOf(reqOpts.uri));
}
reqOpts.uri = uriComponents
.filter(x => x.trim()) // Limit to non-empty strings.
.map(uriComponent => {
const trimSlashesRegex = /^\/*|\/*$/g;
return uriComponent.replace(trimSlashesRegex, '');
})
.join('/');
const childInterceptors = Array.isArray(reqOpts.interceptors_)
? reqOpts.interceptors_
: [];
const localInterceptors = [].slice.call(this.interceptors);
reqOpts.interceptors_ = childInterceptors.concat(localInterceptors);
if (reqOpts.shouldReturnStream) {
return this.parent.requestStream(reqOpts);
}
this.parent.request(reqOpts, callback);
}
request(reqOpts, callback) {
this.request_(reqOpts, callback);
}
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts) {
const opts = { ...reqOpts, shouldReturnStream: true };
return this.request_(opts);
}
}
exports.ServiceObject = ServiceObject;
(0, promisify_1.promisifyAll)(ServiceObject, { exclude: ['getRequestInterceptors'] });

View File

@@ -0,0 +1,125 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { AuthClient, GoogleAuth, GoogleAuthOptions } from 'google-auth-library';
import * as r from 'teeny-request';
import { Interceptor } from './service-object.js';
import { BodyResponseCallback, DecorateRequestOptions, MakeAuthenticatedRequest, PackageJson } from './util.js';
export declare const DEFAULT_PROJECT_ID_TOKEN = "{{projectId}}";
export interface StreamRequestOptions extends DecorateRequestOptions {
shouldReturnStream: true;
}
export interface ServiceConfig {
/**
* The base URL to make API requests to.
*/
baseUrl: string;
/**
* The API Endpoint to use when connecting to the service.
* Example: storage.googleapis.com
*/
apiEndpoint: string;
/**
* The scopes required for the request.
*/
scopes: string[];
projectIdRequired?: boolean;
packageJson: PackageJson;
/**
* Reuse an existing `AuthClient` or `GoogleAuth` client instead of creating a new one.
*/
authClient?: AuthClient | GoogleAuth;
/**
* Set to true if the endpoint is a custom URL
*/
customEndpoint?: boolean;
}
export interface ServiceOptions extends Omit<GoogleAuthOptions, 'authClient'> {
authClient?: AuthClient | GoogleAuth;
interceptors_?: Interceptor[];
email?: string;
token?: string;
timeout?: number;
userAgent?: string;
useAuthWithCustomEndpoint?: boolean;
}
export declare class Service {
baseUrl: string;
private globalInterceptors;
interceptors: Interceptor[];
private packageJson;
projectId: string;
private projectIdRequired;
providedUserAgent?: string;
makeAuthenticatedRequest: MakeAuthenticatedRequest;
authClient: GoogleAuth<AuthClient>;
apiEndpoint: string;
timeout?: number;
universeDomain: string;
customEndpoint: boolean;
/**
* Service is a base class, meant to be inherited from by a "service," like
* BigQuery or Storage.
*
* This handles making authenticated requests by exposing a `makeReq_`
* function.
*
* @constructor
* @alias module:common/service
*
* @param {object} config - Configuration object.
* @param {string} config.baseUrl - The base URL to make API requests to.
* @param {string[]} config.scopes - The scopes required for the request.
* @param {object=} options - [Configuration object](#/docs).
*/
constructor(config: ServiceConfig, options?: ServiceOptions);
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors(): Function[];
/**
* Get and update the Service's project ID.
*
* @param {function} callback - The callback function.
*/
getProjectId(): Promise<string>;
getProjectId(callback: (err: Error | null, projectId?: string) => void): void;
protected getProjectIdAsync(): Promise<string>;
/**
* Make an authenticated API request.
*
* @private
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
private request_;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts: DecorateRequestOptions): r.Request;
}

View File

@@ -0,0 +1,218 @@
"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
Object.defineProperty(exports, "__esModule", { value: true });
exports.Service = exports.DEFAULT_PROJECT_ID_TOKEN = void 0;
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
const google_auth_library_1 = require("google-auth-library");
const uuid = __importStar(require("uuid"));
const util_js_1 = require("./util.js");
const util_js_2 = require("../util.js");
exports.DEFAULT_PROJECT_ID_TOKEN = '{{projectId}}';
class Service {
/**
* Service is a base class, meant to be inherited from by a "service," like
* BigQuery or Storage.
*
* This handles making authenticated requests by exposing a `makeReq_`
* function.
*
* @constructor
* @alias module:common/service
*
* @param {object} config - Configuration object.
* @param {string} config.baseUrl - The base URL to make API requests to.
* @param {string[]} config.scopes - The scopes required for the request.
* @param {object=} options - [Configuration object](#/docs).
*/
constructor(config, options = {}) {
this.baseUrl = config.baseUrl;
this.apiEndpoint = config.apiEndpoint;
this.timeout = options.timeout;
this.globalInterceptors = Array.isArray(options.interceptors_)
? options.interceptors_
: [];
this.interceptors = [];
this.packageJson = config.packageJson;
this.projectId = options.projectId || exports.DEFAULT_PROJECT_ID_TOKEN;
this.projectIdRequired = config.projectIdRequired !== false;
this.providedUserAgent = options.userAgent;
this.universeDomain = options.universeDomain || google_auth_library_1.DEFAULT_UNIVERSE;
this.customEndpoint = config.customEndpoint || false;
this.makeAuthenticatedRequest = util_js_1.util.makeAuthenticatedRequestFactory({
...config,
projectIdRequired: this.projectIdRequired,
projectId: this.projectId,
authClient: options.authClient || config.authClient,
credentials: options.credentials,
keyFile: options.keyFilename,
email: options.email,
clientOptions: {
universeDomain: options.universeDomain,
...options.clientOptions,
},
});
this.authClient = this.makeAuthenticatedRequest.authClient;
const isCloudFunctionEnv = !!process.env.FUNCTION_NAME;
if (isCloudFunctionEnv) {
this.interceptors.push({
request(reqOpts) {
reqOpts.forever = false;
return reqOpts;
},
});
}
}
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors() {
// Interceptors should be returned in the order they were assigned.
return [].slice
.call(this.globalInterceptors)
.concat(this.interceptors)
.filter(interceptor => typeof interceptor.request === 'function')
.map(interceptor => interceptor.request);
}
getProjectId(callback) {
if (!callback) {
return this.getProjectIdAsync();
}
this.getProjectIdAsync().then(p => callback(null, p), callback);
}
async getProjectIdAsync() {
const projectId = await this.authClient.getProjectId();
if (this.projectId === exports.DEFAULT_PROJECT_ID_TOKEN && projectId) {
this.projectId = projectId;
}
return this.projectId;
}
request_(reqOpts, callback) {
reqOpts = { ...reqOpts, timeout: this.timeout };
const isAbsoluteUrl = reqOpts.uri.indexOf('http') === 0;
const uriComponents = [this.baseUrl];
if (this.projectIdRequired) {
if (reqOpts.projectId) {
uriComponents.push('projects');
uriComponents.push(reqOpts.projectId);
}
else {
uriComponents.push('projects');
uriComponents.push(this.projectId);
}
}
uriComponents.push(reqOpts.uri);
if (isAbsoluteUrl) {
uriComponents.splice(0, uriComponents.indexOf(reqOpts.uri));
}
reqOpts.uri = uriComponents
.map(uriComponent => {
const trimSlashesRegex = /^\/*|\/*$/g;
return uriComponent.replace(trimSlashesRegex, '');
})
.join('/')
// Some URIs have colon separators.
// Bad: https://.../projects/:list
// Good: https://.../projects:list
.replace(/\/:/g, ':');
const requestInterceptors = this.getRequestInterceptors();
const interceptorArray = Array.isArray(reqOpts.interceptors_)
? reqOpts.interceptors_
: [];
interceptorArray.forEach(interceptor => {
if (typeof interceptor.request === 'function') {
requestInterceptors.push(interceptor.request);
}
});
requestInterceptors.forEach(requestInterceptor => {
reqOpts = requestInterceptor(reqOpts);
});
delete reqOpts.interceptors_;
const pkg = this.packageJson;
let userAgent = (0, util_js_2.getUserAgentString)();
if (this.providedUserAgent) {
userAgent = `${this.providedUserAgent} ${userAgent}`;
}
reqOpts.headers = {
...reqOpts.headers,
'User-Agent': userAgent,
'x-goog-api-client': `${(0, util_js_2.getRuntimeTrackingString)()} gccl/${pkg.version}-${(0, util_js_2.getModuleFormat)()} gccl-invocation-id/${uuid.v4()}`,
};
if (reqOpts[util_js_1.GCCL_GCS_CMD_KEY]) {
reqOpts.headers['x-goog-api-client'] +=
` gccl-gcs-cmd/${reqOpts[util_js_1.GCCL_GCS_CMD_KEY]}`;
}
if (reqOpts.shouldReturnStream) {
return this.makeAuthenticatedRequest(reqOpts);
}
else {
this.makeAuthenticatedRequest(reqOpts, callback);
}
}
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
request(reqOpts, callback) {
Service.prototype.request_.call(this, reqOpts, callback);
}
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts) {
const opts = { ...reqOpts, shouldReturnStream: true };
return Service.prototype.request_.call(this, opts);
}
}
exports.Service = Service;

View File

@@ -0,0 +1,333 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { AuthClient, GoogleAuth, GoogleAuthOptions } from 'google-auth-library';
import { CredentialBody } from 'google-auth-library';
import * as r from 'teeny-request';
import { Duplex, DuplexOptions, Readable, Writable } from 'stream';
import { Interceptor } from './service-object.js';
/**
* A unique symbol for providing a `gccl-gcs-cmd` value
* for the `X-Goog-API-Client` header.
*
* E.g. the `V` in `X-Goog-API-Client: gccl-gcs-cmd/V`
**/
export declare const GCCL_GCS_CMD_KEY: unique symbol;
export type ResponseBody = any;
export interface DuplexifyOptions extends DuplexOptions {
autoDestroy?: boolean;
end?: boolean;
}
export interface Duplexify extends Duplex {
readonly destroyed: boolean;
setWritable(writable: Writable | false | null): void;
setReadable(readable: Readable | false | null): void;
}
export interface DuplexifyConstructor {
obj(writable?: Writable | false | null, readable?: Readable | false | null, options?: DuplexifyOptions): Duplexify;
new (writable?: Writable | false | null, readable?: Readable | false | null, options?: DuplexifyOptions): Duplexify;
(writable?: Writable | false | null, readable?: Readable | false | null, options?: DuplexifyOptions): Duplexify;
}
export interface ParsedHttpRespMessage {
resp: r.Response;
err?: ApiError;
}
export interface MakeAuthenticatedRequest {
(reqOpts: DecorateRequestOptions): Duplexify;
(reqOpts: DecorateRequestOptions, options?: MakeAuthenticatedRequestOptions): void | Abortable;
(reqOpts: DecorateRequestOptions, callback?: BodyResponseCallback): void | Abortable;
(reqOpts: DecorateRequestOptions, optionsOrCallback?: MakeAuthenticatedRequestOptions | BodyResponseCallback): void | Abortable | Duplexify;
getCredentials: (callback: (err?: Error | null, credentials?: CredentialBody) => void) => void;
authClient: GoogleAuth<AuthClient>;
}
export interface Abortable {
abort(): void;
}
export type AbortableDuplex = Duplexify & Abortable;
export interface PackageJson {
name: string;
version: string;
}
export interface MakeAuthenticatedRequestFactoryConfig extends Omit<GoogleAuthOptions, 'authClient'> {
/**
* Automatically retry requests if the response is related to rate limits or
* certain intermittent server errors. We will exponentially backoff
* subsequent requests by default. (default: true)
*/
autoRetry?: boolean;
/**
* If true, just return the provided request options. Default: false.
*/
customEndpoint?: boolean;
/**
* If true, will authenticate when using a custom endpoint. Default: false.
*/
useAuthWithCustomEndpoint?: boolean;
/**
* Account email address, required for PEM/P12 usage.
*/
email?: string;
/**
* Maximum number of automatic retries attempted before returning the error.
* (default: 3)
*/
maxRetries?: number;
stream?: Duplexify;
/**
* A pre-instantiated `AuthClient` or `GoogleAuth` client that should be used.
* A new client will be created if this is not set.
*/
authClient?: AuthClient | GoogleAuth;
/**
* Determines if a projectId is required for authenticated requests. Defaults to `true`.
*/
projectIdRequired?: boolean;
}
export interface MakeAuthenticatedRequestOptions {
onAuthenticated: OnAuthenticatedCallback;
}
export interface OnAuthenticatedCallback {
(err: Error | null, reqOpts?: DecorateRequestOptions): void;
}
export interface GoogleErrorBody {
code: number;
errors?: GoogleInnerError[];
response: r.Response;
message?: string;
}
export interface GoogleInnerError {
reason?: string;
message?: string;
}
export interface MakeWritableStreamOptions {
/**
* A connection instance used to get a token with and send the request
* through.
*/
connection?: {};
/**
* Metadata to send at the head of the request.
*/
metadata?: {
contentType?: string;
};
/**
* Request object, in the format of a standard Node.js http.request() object.
*/
request?: r.Options;
makeAuthenticatedRequest(reqOpts: r.OptionsWithUri & {
[GCCL_GCS_CMD_KEY]?: string;
}, fnobj: {
onAuthenticated(err: Error | null, authenticatedReqOpts?: r.Options): void;
}): void;
}
export interface DecorateRequestOptions extends r.CoreOptions {
autoPaginate?: boolean;
autoPaginateVal?: boolean;
objectMode?: boolean;
maxRetries?: number;
uri: string;
interceptors_?: Interceptor[];
shouldReturnStream?: boolean;
projectId?: string;
[GCCL_GCS_CMD_KEY]?: string;
}
export interface ParsedHttpResponseBody {
body: ResponseBody;
err?: Error;
}
/**
* Custom error type for API errors.
*
* @param {object} errorBody - Error object.
*/
export declare class ApiError extends Error {
code?: number;
errors?: GoogleInnerError[];
response?: r.Response;
constructor(errorMessage: string);
constructor(errorBody: GoogleErrorBody);
/**
* Pieces together an error message by combining all unique error messages
* returned from a single GoogleError
*
* @private
*
* @param {GoogleErrorBody} err The original error.
* @param {GoogleInnerError[]} [errors] Inner errors, if any.
* @returns {string}
*/
static createMultiErrorMessage(err: GoogleErrorBody, errors?: GoogleInnerError[]): string;
}
/**
* Custom error type for partial errors returned from the API.
*
* @param {object} b - Error object.
*/
export declare class PartialFailureError extends Error {
errors?: GoogleInnerError[];
response?: r.Response;
constructor(b: GoogleErrorBody);
}
export interface BodyResponseCallback {
(err: Error | ApiError | null, body?: ResponseBody, res?: r.Response): void;
}
export interface RetryOptions {
retryDelayMultiplier?: number;
totalTimeout?: number;
maxRetryDelay?: number;
autoRetry?: boolean;
maxRetries?: number;
retryableErrorFn?: (err: ApiError) => boolean;
}
export interface MakeRequestConfig {
/**
* Automatically retry requests if the response is related to rate limits or
* certain intermittent server errors. We will exponentially backoff
* subsequent requests by default. (default: true)
*/
autoRetry?: boolean;
/**
* Maximum number of automatic retries attempted before returning the error.
* (default: 3)
*/
maxRetries?: number;
retries?: number;
retryOptions?: RetryOptions;
stream?: Duplexify;
shouldRetryFn?: (response?: r.Response) => boolean;
}
export declare class Util {
ApiError: typeof ApiError;
PartialFailureError: typeof PartialFailureError;
/**
* No op.
*
* @example
* function doSomething(callback) {
* callback = callback || noop;
* }
*/
noop(): void;
/**
* Uniformly process an API response.
*
* @param {*} err - Error value.
* @param {*} resp - Response value.
* @param {*} body - Body value.
* @param {function} callback - The callback function.
*/
handleResp(err: Error | null, resp?: r.Response | null, body?: ResponseBody, callback?: BodyResponseCallback): void;
/**
* Sniff an incoming HTTP response message for errors.
*
* @param {object} httpRespMessage - An incoming HTTP response message from `request`.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.resp - The original response object.
*/
parseHttpRespMessage(httpRespMessage: r.Response): ParsedHttpRespMessage;
/**
* Parse the response body from an HTTP request.
*
* @param {object} body - The response body.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.body - The original body value provided
* will try to be JSON.parse'd. If it's successful, the parsed value will
* be returned here, otherwise the original value and an error will be returned.
*/
parseHttpRespBody(body: ResponseBody): ParsedHttpResponseBody;
/**
* Take a Duplexify stream, fetch an authenticated connection header, and
* create an outgoing writable stream.
*
* @param {Duplexify} dup - Duplexify stream.
* @param {object} options - Configuration object.
* @param {module:common/connection} options.connection - A connection instance used to get a token with and send the request through.
* @param {object} options.metadata - Metadata to send at the head of the request.
* @param {object} options.request - Request object, in the format of a standard Node.js http.request() object.
* @param {string=} options.request.method - Default: "POST".
* @param {string=} options.request.qs.uploadType - Default: "multipart".
* @param {string=} options.streamContentType - Default: "application/octet-stream".
* @param {function} onComplete - Callback, executed after the writable Request stream has completed.
*/
makeWritableStream(dup: Duplexify, options: MakeWritableStreamOptions, onComplete?: Function): void;
/**
* Returns true if the API request should be retried, given the error that was
* given the first time the request was attempted. This is used for rate limit
* related errors as well as intermittent server errors.
*
* @param {error} err - The API error to check if it is appropriate to retry.
* @return {boolean} True if the API request should be retried, false otherwise.
*/
shouldRetryRequest(err?: ApiError): boolean;
/**
* Get a function for making authenticated requests.
*
* @param {object} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {object=} config.credentials - Credentials object.
* @param {boolean=} config.customEndpoint - If true, just return the provided request options. Default: false.
* @param {boolean=} config.useAuthWithCustomEndpoint - If true, will authenticate when using a custom endpoint. Default: false.
* @param {string=} config.email - Account email address, required for PEM/P12 usage.
* @param {number=} config.maxRetries - Maximum number of automatic retries attempted before returning the error. (default: 3)
* @param {string=} config.keyFile - Path to a .json, .pem, or .p12 keyfile.
* @param {array} config.scopes - Array of scopes required for the API.
*/
makeAuthenticatedRequestFactory(config: MakeAuthenticatedRequestFactoryConfig): MakeAuthenticatedRequest;
/**
* Make a request through the `retryRequest` module with built-in error
* handling and exponential back off.
*
* @param {object} reqOpts - Request options in the format `request` expects.
* @param {object=} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {number=} config.maxRetries - Maximum number of automatic retries
* attempted before returning the error. (default: 3)
* @param {object=} config.request - HTTP module for request calls.
* @param {function} callback - The callback function.
*/
makeRequest(reqOpts: DecorateRequestOptions, config: MakeRequestConfig, callback: BodyResponseCallback): void | Abortable;
/**
* Decorate the options about to be made in a request.
*
* @param {object} reqOpts - The options to be passed to `request`.
* @param {string} projectId - The project ID.
* @return {object} reqOpts - The decorated reqOpts.
*/
decorateRequest(reqOpts: DecorateRequestOptions, projectId: string): DecorateRequestOptions;
isCustomType(unknown: any, module: string): boolean;
/**
* Given two parameters, figure out if this is either:
* - Just a callback function
* - An options object, and then a callback function
* @param optionsOrCallback An options object or callback.
* @param cb A potentially undefined callback.
*/
maybeOptionsOrCallback<T = {}, C = (err?: Error) => void>(optionsOrCallback?: T | C, cb?: C): [T, C];
_getDefaultHeaders(gcclGcsCmd?: string): {
'User-Agent': string;
'x-goog-api-client': string;
};
}
declare const util: Util;
export { util };

View File

@@ -0,0 +1,711 @@
"use strict";
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.util = exports.Util = exports.PartialFailureError = exports.ApiError = exports.GCCL_GCS_CMD_KEY = void 0;
/*!
* @module common/util
*/
const projectify_1 = require("@google-cloud/projectify");
const htmlEntities = __importStar(require("html-entities"));
const google_auth_library_1 = require("google-auth-library");
const retry_request_1 = __importDefault(require("retry-request"));
const stream_1 = require("stream");
const teeny_request_1 = require("teeny-request");
const uuid = __importStar(require("uuid"));
const service_js_1 = require("./service.js");
const util_js_1 = require("../util.js");
const duplexify_1 = __importDefault(require("duplexify"));
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
const package_json_helper_cjs_1 = require("../package-json-helper.cjs");
const packageJson = (0, package_json_helper_cjs_1.getPackageJSON)();
/**
* A unique symbol for providing a `gccl-gcs-cmd` value
* for the `X-Goog-API-Client` header.
*
* E.g. the `V` in `X-Goog-API-Client: gccl-gcs-cmd/V`
**/
exports.GCCL_GCS_CMD_KEY = Symbol.for('GCCL_GCS_CMD');
const requestDefaults = {
timeout: 60000,
gzip: true,
forever: true,
pool: {
maxSockets: Infinity,
},
};
/**
* Default behavior: Automatically retry retriable server errors.
*
* @const {boolean}
* @private
*/
const AUTO_RETRY_DEFAULT = true;
/**
* Default behavior: Only attempt to retry retriable errors 3 times.
*
* @const {number}
* @private
*/
const MAX_RETRY_DEFAULT = 3;
/**
* Custom error type for API errors.
*
* @param {object} errorBody - Error object.
*/
class ApiError extends Error {
constructor(errorBodyOrMessage) {
super();
if (typeof errorBodyOrMessage !== 'object') {
this.message = errorBodyOrMessage || '';
return;
}
const errorBody = errorBodyOrMessage;
this.code = errorBody.code;
this.errors = errorBody.errors;
this.response = errorBody.response;
try {
this.errors = JSON.parse(this.response.body).error.errors;
}
catch (e) {
this.errors = errorBody.errors;
}
this.message = ApiError.createMultiErrorMessage(errorBody, this.errors);
Error.captureStackTrace(this);
}
/**
* Pieces together an error message by combining all unique error messages
* returned from a single GoogleError
*
* @private
*
* @param {GoogleErrorBody} err The original error.
* @param {GoogleInnerError[]} [errors] Inner errors, if any.
* @returns {string}
*/
static createMultiErrorMessage(err, errors) {
const messages = new Set();
if (err.message) {
messages.add(err.message);
}
if (errors && errors.length) {
errors.forEach(({ message }) => messages.add(message));
}
else if (err.response && err.response.body) {
messages.add(htmlEntities.decode(err.response.body.toString()));
}
else if (!err.message) {
messages.add('A failure occurred during this request.');
}
let messageArr = Array.from(messages);
if (messageArr.length > 1) {
messageArr = messageArr.map((message, i) => ` ${i + 1}. ${message}`);
messageArr.unshift('Multiple errors occurred during the request. Please see the `errors` array for complete details.\n');
messageArr.push('\n');
}
return messageArr.join('\n');
}
}
exports.ApiError = ApiError;
/**
* Custom error type for partial errors returned from the API.
*
* @param {object} b - Error object.
*/
class PartialFailureError extends Error {
constructor(b) {
super();
const errorObject = b;
this.errors = errorObject.errors;
this.name = 'PartialFailureError';
this.response = errorObject.response;
this.message = ApiError.createMultiErrorMessage(errorObject, this.errors);
}
}
exports.PartialFailureError = PartialFailureError;
class Util {
constructor() {
this.ApiError = ApiError;
this.PartialFailureError = PartialFailureError;
}
/**
* No op.
*
* @example
* function doSomething(callback) {
* callback = callback || noop;
* }
*/
noop() { }
/**
* Uniformly process an API response.
*
* @param {*} err - Error value.
* @param {*} resp - Response value.
* @param {*} body - Body value.
* @param {function} callback - The callback function.
*/
handleResp(err, resp, body, callback) {
callback = callback || util.noop;
const parsedResp = {
err: err || null,
...(resp && util.parseHttpRespMessage(resp)),
...(body && util.parseHttpRespBody(body)),
};
// Assign the parsed body to resp.body, even if { json: false } was passed
// as a request option.
// We assume that nobody uses the previously unparsed value of resp.body.
if (!parsedResp.err && resp && typeof parsedResp.body === 'object') {
parsedResp.resp.body = parsedResp.body;
}
if (parsedResp.err && resp) {
parsedResp.err.response = resp;
}
callback(parsedResp.err, parsedResp.body, parsedResp.resp);
}
/**
* Sniff an incoming HTTP response message for errors.
*
* @param {object} httpRespMessage - An incoming HTTP response message from `request`.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.resp - The original response object.
*/
parseHttpRespMessage(httpRespMessage) {
const parsedHttpRespMessage = {
resp: httpRespMessage,
};
if (httpRespMessage.statusCode < 200 || httpRespMessage.statusCode > 299) {
// Unknown error. Format according to ApiError standard.
parsedHttpRespMessage.err = new ApiError({
errors: new Array(),
code: httpRespMessage.statusCode,
message: httpRespMessage.statusMessage,
response: httpRespMessage,
});
}
return parsedHttpRespMessage;
}
/**
* Parse the response body from an HTTP request.
*
* @param {object} body - The response body.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.body - The original body value provided
* will try to be JSON.parse'd. If it's successful, the parsed value will
* be returned here, otherwise the original value and an error will be returned.
*/
parseHttpRespBody(body) {
const parsedHttpRespBody = {
body,
};
if (typeof body === 'string') {
try {
parsedHttpRespBody.body = JSON.parse(body);
}
catch (err) {
parsedHttpRespBody.body = body;
}
}
if (parsedHttpRespBody.body && parsedHttpRespBody.body.error) {
// Error from JSON API.
parsedHttpRespBody.err = new ApiError(parsedHttpRespBody.body.error);
}
return parsedHttpRespBody;
}
/**
* Take a Duplexify stream, fetch an authenticated connection header, and
* create an outgoing writable stream.
*
* @param {Duplexify} dup - Duplexify stream.
* @param {object} options - Configuration object.
* @param {module:common/connection} options.connection - A connection instance used to get a token with and send the request through.
* @param {object} options.metadata - Metadata to send at the head of the request.
* @param {object} options.request - Request object, in the format of a standard Node.js http.request() object.
* @param {string=} options.request.method - Default: "POST".
* @param {string=} options.request.qs.uploadType - Default: "multipart".
* @param {string=} options.streamContentType - Default: "application/octet-stream".
* @param {function} onComplete - Callback, executed after the writable Request stream has completed.
*/
makeWritableStream(dup, options, onComplete) {
var _a;
onComplete = onComplete || util.noop;
const writeStream = new ProgressStream();
writeStream.on('progress', evt => dup.emit('progress', evt));
dup.setWritable(writeStream);
const defaultReqOpts = {
method: 'POST',
qs: {
uploadType: 'multipart',
},
timeout: 0,
maxRetries: 0,
};
const metadata = options.metadata || {};
const reqOpts = {
...defaultReqOpts,
...options.request,
qs: {
...defaultReqOpts.qs,
...(_a = options.request) === null || _a === void 0 ? void 0 : _a.qs,
},
multipart: [
{
'Content-Type': 'application/json',
body: JSON.stringify(metadata),
},
{
'Content-Type': metadata.contentType || 'application/octet-stream',
body: writeStream,
},
],
};
options.makeAuthenticatedRequest(reqOpts, {
onAuthenticated(err, authenticatedReqOpts) {
if (err) {
dup.destroy(err);
return;
}
requestDefaults.headers = util._getDefaultHeaders(reqOpts[exports.GCCL_GCS_CMD_KEY]);
const request = teeny_request_1.teenyRequest.defaults(requestDefaults);
request(authenticatedReqOpts, (err, resp, body) => {
util.handleResp(err, resp, body, (err, data) => {
if (err) {
dup.destroy(err);
return;
}
dup.emit('response', resp);
onComplete(data);
});
});
},
});
}
/**
* Returns true if the API request should be retried, given the error that was
* given the first time the request was attempted. This is used for rate limit
* related errors as well as intermittent server errors.
*
* @param {error} err - The API error to check if it is appropriate to retry.
* @return {boolean} True if the API request should be retried, false otherwise.
*/
shouldRetryRequest(err) {
if (err) {
if ([408, 429, 500, 502, 503, 504].indexOf(err.code) !== -1) {
return true;
}
if (err.errors) {
for (const e of err.errors) {
const reason = e.reason;
if (reason === 'rateLimitExceeded') {
return true;
}
if (reason === 'userRateLimitExceeded') {
return true;
}
if (reason && reason.includes('EAI_AGAIN')) {
return true;
}
}
}
}
return false;
}
/**
* Get a function for making authenticated requests.
*
* @param {object} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {object=} config.credentials - Credentials object.
* @param {boolean=} config.customEndpoint - If true, just return the provided request options. Default: false.
* @param {boolean=} config.useAuthWithCustomEndpoint - If true, will authenticate when using a custom endpoint. Default: false.
* @param {string=} config.email - Account email address, required for PEM/P12 usage.
* @param {number=} config.maxRetries - Maximum number of automatic retries attempted before returning the error. (default: 3)
* @param {string=} config.keyFile - Path to a .json, .pem, or .p12 keyfile.
* @param {array} config.scopes - Array of scopes required for the API.
*/
makeAuthenticatedRequestFactory(config) {
const googleAutoAuthConfig = { ...config };
if (googleAutoAuthConfig.projectId === service_js_1.DEFAULT_PROJECT_ID_TOKEN) {
delete googleAutoAuthConfig.projectId;
}
let authClient;
if (googleAutoAuthConfig.authClient instanceof google_auth_library_1.GoogleAuth) {
// Use an existing `GoogleAuth`
authClient = googleAutoAuthConfig.authClient;
}
else {
// Pass an `AuthClient` & `clientOptions` to `GoogleAuth`, if available
authClient = new google_auth_library_1.GoogleAuth({
...googleAutoAuthConfig,
authClient: googleAutoAuthConfig.authClient,
clientOptions: googleAutoAuthConfig.clientOptions,
});
}
function makeAuthenticatedRequest(reqOpts, optionsOrCallback) {
let stream;
let projectId;
const reqConfig = { ...config };
let activeRequest_;
if (!optionsOrCallback) {
stream = (0, duplexify_1.default)();
reqConfig.stream = stream;
}
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : undefined;
const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : undefined;
async function setProjectId() {
projectId = await authClient.getProjectId();
}
const onAuthenticated = async (err, authenticatedReqOpts) => {
const authLibraryError = err;
const autoAuthFailed = err &&
typeof err.message === 'string' &&
err.message.indexOf('Could not load the default credentials') > -1;
if (autoAuthFailed) {
// Even though authentication failed, the API might not actually
// care.
authenticatedReqOpts = reqOpts;
}
if (!err || autoAuthFailed) {
try {
// Try with existing `projectId` value
authenticatedReqOpts = util.decorateRequest(authenticatedReqOpts, projectId);
err = null;
}
catch (e) {
if (e instanceof projectify_1.MissingProjectIdError) {
// A `projectId` was required, but we don't have one.
try {
// Attempt to get the `projectId`
await setProjectId();
authenticatedReqOpts = util.decorateRequest(authenticatedReqOpts, projectId);
err = null;
}
catch (e) {
// Re-use the "Could not load the default credentials error" if
// auto auth failed.
err = err || e;
}
}
else {
// Some other error unrelated to missing `projectId`
err = err || e;
}
}
}
if (err) {
if (stream) {
stream.destroy(err);
}
else {
const fn = options && options.onAuthenticated
? options.onAuthenticated
: callback;
fn(err);
}
return;
}
if (options && options.onAuthenticated) {
options.onAuthenticated(null, authenticatedReqOpts);
}
else {
activeRequest_ = util.makeRequest(authenticatedReqOpts, reqConfig, (apiResponseError, ...params) => {
if (apiResponseError &&
apiResponseError.code === 401 &&
authLibraryError) {
// Re-use the "Could not load the default credentials error" if
// the API request failed due to missing credentials.
apiResponseError = authLibraryError;
}
callback(apiResponseError, ...params);
});
}
};
const prepareRequest = async () => {
try {
const getProjectId = async () => {
if (config.projectId &&
config.projectId !== service_js_1.DEFAULT_PROJECT_ID_TOKEN) {
// The user provided a project ID. We don't need to check with the
// auth client, it could be incorrect.
return config.projectId;
}
if (config.projectIdRequired === false) {
// A projectId is not required. Return the default.
return service_js_1.DEFAULT_PROJECT_ID_TOKEN;
}
return setProjectId();
};
const authorizeRequest = async () => {
if (reqConfig.customEndpoint &&
!reqConfig.useAuthWithCustomEndpoint) {
// Using a custom API override. Do not use `google-auth-library` for
// authentication. (ex: connecting to a local Datastore server)
return reqOpts;
}
else {
return authClient.authorizeRequest(reqOpts);
}
};
const [_projectId, authorizedReqOpts] = await Promise.all([
getProjectId(),
authorizeRequest(),
]);
if (_projectId) {
projectId = _projectId;
}
return onAuthenticated(null, authorizedReqOpts);
}
catch (e) {
return onAuthenticated(e);
}
};
prepareRequest();
if (stream) {
return stream;
}
return {
abort() {
setImmediate(() => {
if (activeRequest_) {
activeRequest_.abort();
activeRequest_ = null;
}
});
},
};
}
const mar = makeAuthenticatedRequest;
mar.getCredentials = authClient.getCredentials.bind(authClient);
mar.authClient = authClient;
return mar;
}
/**
* Make a request through the `retryRequest` module with built-in error
* handling and exponential back off.
*
* @param {object} reqOpts - Request options in the format `request` expects.
* @param {object=} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {number=} config.maxRetries - Maximum number of automatic retries
* attempted before returning the error. (default: 3)
* @param {object=} config.request - HTTP module for request calls.
* @param {function} callback - The callback function.
*/
makeRequest(reqOpts, config, callback) {
var _a, _b, _c, _d, _e;
let autoRetryValue = AUTO_RETRY_DEFAULT;
if (config.autoRetry !== undefined) {
autoRetryValue = config.autoRetry;
}
else if (((_a = config.retryOptions) === null || _a === void 0 ? void 0 : _a.autoRetry) !== undefined) {
autoRetryValue = config.retryOptions.autoRetry;
}
let maxRetryValue = MAX_RETRY_DEFAULT;
if (config.maxRetries !== undefined) {
maxRetryValue = config.maxRetries;
}
else if (((_b = config.retryOptions) === null || _b === void 0 ? void 0 : _b.maxRetries) !== undefined) {
maxRetryValue = config.retryOptions.maxRetries;
}
requestDefaults.headers = this._getDefaultHeaders(reqOpts[exports.GCCL_GCS_CMD_KEY]);
const options = {
request: teeny_request_1.teenyRequest.defaults(requestDefaults),
retries: autoRetryValue !== false ? maxRetryValue : 0,
noResponseRetries: autoRetryValue !== false ? maxRetryValue : 0,
shouldRetryFn(httpRespMessage) {
var _a, _b;
const err = util.parseHttpRespMessage(httpRespMessage).err;
if ((_a = config.retryOptions) === null || _a === void 0 ? void 0 : _a.retryableErrorFn) {
return err && ((_b = config.retryOptions) === null || _b === void 0 ? void 0 : _b.retryableErrorFn(err));
}
return err && util.shouldRetryRequest(err);
},
maxRetryDelay: (_c = config.retryOptions) === null || _c === void 0 ? void 0 : _c.maxRetryDelay,
retryDelayMultiplier: (_d = config.retryOptions) === null || _d === void 0 ? void 0 : _d.retryDelayMultiplier,
totalTimeout: (_e = config.retryOptions) === null || _e === void 0 ? void 0 : _e.totalTimeout,
};
if (typeof reqOpts.maxRetries === 'number') {
options.retries = reqOpts.maxRetries;
options.noResponseRetries = reqOpts.maxRetries;
}
if (!config.stream) {
return (0, retry_request_1.default)(reqOpts, options,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(err, response, body) => {
util.handleResp(err, response, body, callback);
});
}
const dup = config.stream;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let requestStream;
const isGetRequest = (reqOpts.method || 'GET').toUpperCase() === 'GET';
if (isGetRequest) {
requestStream = (0, retry_request_1.default)(reqOpts, options);
dup.setReadable(requestStream);
}
else {
// Streaming writable HTTP requests cannot be retried.
requestStream = options.request(reqOpts);
dup.setWritable(requestStream);
}
// Replay the Request events back to the stream.
requestStream
.on('error', dup.destroy.bind(dup))
.on('response', dup.emit.bind(dup, 'response'))
.on('complete', dup.emit.bind(dup, 'complete'));
dup.abort = requestStream.abort;
return dup;
}
/**
* Decorate the options about to be made in a request.
*
* @param {object} reqOpts - The options to be passed to `request`.
* @param {string} projectId - The project ID.
* @return {object} reqOpts - The decorated reqOpts.
*/
decorateRequest(reqOpts, projectId) {
delete reqOpts.autoPaginate;
delete reqOpts.autoPaginateVal;
delete reqOpts.objectMode;
if (reqOpts.qs !== null && typeof reqOpts.qs === 'object') {
delete reqOpts.qs.autoPaginate;
delete reqOpts.qs.autoPaginateVal;
reqOpts.qs = (0, projectify_1.replaceProjectIdToken)(reqOpts.qs, projectId);
}
if (Array.isArray(reqOpts.multipart)) {
reqOpts.multipart = reqOpts.multipart.map(part => {
return (0, projectify_1.replaceProjectIdToken)(part, projectId);
});
}
if (reqOpts.json !== null && typeof reqOpts.json === 'object') {
delete reqOpts.json.autoPaginate;
delete reqOpts.json.autoPaginateVal;
reqOpts.json = (0, projectify_1.replaceProjectIdToken)(reqOpts.json, projectId);
}
reqOpts.uri = (0, projectify_1.replaceProjectIdToken)(reqOpts.uri, projectId);
return reqOpts;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
isCustomType(unknown, module) {
function getConstructorName(obj) {
return obj.constructor && obj.constructor.name.toLowerCase();
}
const moduleNameParts = module.split('/');
const parentModuleName = moduleNameParts[0] && moduleNameParts[0].toLowerCase();
const subModuleName = moduleNameParts[1] && moduleNameParts[1].toLowerCase();
if (subModuleName && getConstructorName(unknown) !== subModuleName) {
return false;
}
let walkingModule = unknown;
// eslint-disable-next-line no-constant-condition
while (true) {
if (getConstructorName(walkingModule) === parentModuleName) {
return true;
}
walkingModule = walkingModule.parent;
if (!walkingModule) {
return false;
}
}
}
/**
* Given two parameters, figure out if this is either:
* - Just a callback function
* - An options object, and then a callback function
* @param optionsOrCallback An options object or callback.
* @param cb A potentially undefined callback.
*/
maybeOptionsOrCallback(optionsOrCallback, cb) {
return typeof optionsOrCallback === 'function'
? [{}, optionsOrCallback]
: [optionsOrCallback, cb];
}
_getDefaultHeaders(gcclGcsCmd) {
const headers = {
'User-Agent': (0, util_js_1.getUserAgentString)(),
'x-goog-api-client': `${(0, util_js_1.getRuntimeTrackingString)()} gccl/${packageJson.version}-${(0, util_js_1.getModuleFormat)()} gccl-invocation-id/${uuid.v4()}`,
};
if (gcclGcsCmd) {
headers['x-goog-api-client'] += ` gccl-gcs-cmd/${gcclGcsCmd}`;
}
return headers;
}
}
exports.Util = Util;
/**
* Basic Passthrough Stream that records the number of bytes read
* every time the cursor is moved.
*/
class ProgressStream extends stream_1.Transform {
constructor() {
super(...arguments);
this.bytesRead = 0;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
_transform(chunk, encoding, callback) {
this.bytesRead += chunk.length;
this.emit('progress', { bytesWritten: this.bytesRead, contentLength: '*' });
this.push(chunk);
callback();
}
}
const util = new Util();
exports.util = util;

View File

@@ -0,0 +1,106 @@
import { BaseMetadata, ServiceObject } from './nodejs-common/index.js';
import { ResponseBody } from './nodejs-common/util.js';
import { Bucket } from './bucket.js';
export interface DeleteNotificationOptions {
userProject?: string;
}
export interface GetNotificationMetadataOptions {
userProject?: string;
}
/**
* @typedef {array} GetNotificationMetadataResponse
* @property {object} 0 The notification metadata.
* @property {object} 1 The full API response.
*/
export type GetNotificationMetadataResponse = [ResponseBody, unknown];
/**
* @callback GetNotificationMetadataCallback
* @param {?Error} err Request error, if any.
* @param {object} files The notification metadata.
* @param {object} apiResponse The full API response.
*/
export interface GetNotificationMetadataCallback {
(err: Error | null, metadata?: ResponseBody, apiResponse?: unknown): void;
}
/**
* @typedef {array} GetNotificationResponse
* @property {Notification} 0 The {@link Notification}
* @property {object} 1 The full API response.
*/
export type GetNotificationResponse = [Notification, unknown];
export interface GetNotificationOptions {
/**
* Automatically create the object if it does not exist. Default: `false`.
*/
autoCreate?: boolean;
/**
* The ID of the project which will be billed for the request.
*/
userProject?: string;
}
/**
* @callback GetNotificationCallback
* @param {?Error} err Request error, if any.
* @param {Notification} notification The {@link Notification}.
* @param {object} apiResponse The full API response.
*/
export interface GetNotificationCallback {
(err: Error | null, notification?: Notification | null, apiResponse?: unknown): void;
}
/**
* @callback DeleteNotificationCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
export interface DeleteNotificationCallback {
(err: Error | null, apiResponse?: unknown): void;
}
export interface NotificationMetadata extends BaseMetadata {
custom_attributes?: {
[key: string]: string;
};
event_types?: string[];
object_name_prefix?: string;
payload_format?: 'JSON_API_V1' | 'NONE';
topic?: string;
}
/**
* The API-formatted resource description of the notification.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name Notification#metadata
* @type {object}
*/
/**
* A Notification object is created from your {@link Bucket} object using
* {@link Bucket#notification}. Use it to interact with Cloud Pub/Sub
* notifications.
*
* See {@link https://cloud.google.com/storage/docs/pubsub-notifications| Cloud Pub/Sub Notifications for Google Cloud Storage}
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket The bucket instance this notification is attached to.
* @param {string} id The ID of the notification.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const notification = myBucket.notification('1');
* ```
*/
declare class Notification extends ServiceObject<Notification, NotificationMetadata> {
constructor(bucket: Bucket, id: string);
}
/**
* Reference to the {@link Notification} class.
* @name module:@google-cloud/storage.Notification
* @see Notification
*/
export { Notification };

View File

@@ -0,0 +1,265 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Object.defineProperty(exports, "__esModule", { value: true });
exports.Notification = void 0;
const index_js_1 = require("./nodejs-common/index.js");
const promisify_1 = require("@google-cloud/promisify");
/**
* The API-formatted resource description of the notification.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name Notification#metadata
* @type {object}
*/
/**
* A Notification object is created from your {@link Bucket} object using
* {@link Bucket#notification}. Use it to interact with Cloud Pub/Sub
* notifications.
*
* See {@link https://cloud.google.com/storage/docs/pubsub-notifications| Cloud Pub/Sub Notifications for Google Cloud Storage}
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket The bucket instance this notification is attached to.
* @param {string} id The ID of the notification.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const notification = myBucket.notification('1');
* ```
*/
class Notification extends index_js_1.ServiceObject {
constructor(bucket, id) {
const requestQueryObject = {};
const methods = {
/**
* Creates a notification subscription for the bucket.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/insert| Notifications: insert}
* @method Notification#create
*
* @param {Topic|string} topic The Cloud PubSub topic to which this
* subscription publishes. If the project ID is omitted, the current
* project ID will be used.
*
* Acceptable formats are:
* - `projects/grape-spaceship-123/topics/my-topic`
*
* - `my-topic`
* @param {CreateNotificationRequest} [options] Metadata to set for
* the notification.
* @param {CreateNotificationCallback} [callback] Callback function.
* @returns {Promise<CreateNotificationResponse>}
* @throws {Error} If a valid topic is not provided.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.create(function(err, notification, apiResponse) {
* if (!err) {
* // The notification was created successfully.
* }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.create().then(function(data) {
* const notification = data[0];
* const apiResponse = data[1];
* });
* ```
*/
create: true,
/**
* @typedef {array} DeleteNotificationResponse
* @property {object} 0 The full API response.
*/
/**
* Permanently deletes a notification subscription.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/delete| Notifications: delete API Documentation}
*
* @param {object} [options] Configuration options.
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {DeleteNotificationCallback} [callback] Callback function.
* @returns {Promise<DeleteNotificationResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.delete(function(err, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.delete().then(function(data) {
* const apiResponse = data[0];
* });
*
* ```
* @example <caption>include:samples/deleteNotification.js</caption>
* region_tag:storage_delete_bucket_notification
* Another example:
*/
delete: {
reqOpts: {
qs: requestQueryObject,
},
},
/**
* Get a notification and its metadata if it exists.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/get| Notifications: get API Documentation}
*
* @param {object} [options] Configuration options.
* See {@link Bucket#createNotification} for create options.
* @param {boolean} [options.autoCreate] Automatically create the object if
* it does not exist. Default: `false`.
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {GetNotificationCallback} [callback] Callback function.
* @return {Promise<GetNotificationCallback>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.get(function(err, notification, apiResponse) {
* // `notification.metadata` has been populated.
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.get().then(function(data) {
* const notification = data[0];
* const apiResponse = data[1];
* });
* ```
*/
get: {
reqOpts: {
qs: requestQueryObject,
},
},
/**
* Get the notification's metadata.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/get| Notifications: get API Documentation}
*
* @param {object} [options] Configuration options.
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {GetNotificationMetadataCallback} [callback] Callback function.
* @returns {Promise<GetNotificationMetadataResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.getMetadata(function(err, metadata, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.getMetadata().then(function(data) {
* const metadata = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/getMetadataNotifications.js</caption>
* region_tag:storage_print_pubsub_bucket_notification
* Another example:
*/
getMetadata: {
reqOpts: {
qs: requestQueryObject,
},
},
/**
* @typedef {array} NotificationExistsResponse
* @property {boolean} 0 Whether the notification exists or not.
*/
/**
* @callback NotificationExistsCallback
* @param {?Error} err Request error, if any.
* @param {boolean} exists Whether the notification exists or not.
*/
/**
* Check if the notification exists.
*
* @method Notification#exists
* @param {NotificationExistsCallback} [callback] Callback function.
* @returns {Promise<NotificationExistsResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.exists(function(err, exists) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.exists().then(function(data) {
* const exists = data[0];
* });
* ```
*/
exists: true,
};
super({
parent: bucket,
baseUrl: '/notificationConfigs',
id: id.toString(),
createMethod: bucket.createNotification.bind(bucket),
methods,
});
}
}
exports.Notification = Notification;
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
(0, promisify_1.promisifyAll)(Notification);

View File

@@ -0,0 +1,21 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/* eslint-disable node/no-missing-require */
function getPackageJSON() {
return require('../../../package.json');
}
exports.getPackageJSON = getPackageJSON;

View File

@@ -0,0 +1,337 @@
import { GaxiosOptions, GaxiosPromise, GaxiosResponse } from 'gaxios';
import { GoogleAuthOptions } from 'google-auth-library';
import { Writable, WritableOptions } from 'stream';
import { RetryOptions, PreconditionOptions } from './storage.js';
import { GCCL_GCS_CMD_KEY } from './nodejs-common/util.js';
import { FileMetadata } from './file.js';
export declare const PROTOCOL_REGEX: RegExp;
export interface ErrorWithCode extends Error {
code: number;
status?: number | string;
}
export type CreateUriCallback = (err: Error | null, uri?: string) => void;
export interface Encryption {
key: {};
hash: {};
}
export type PredefinedAcl = 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
export interface QueryParameters extends PreconditionOptions {
contentEncoding?: string;
kmsKeyName?: string;
predefinedAcl?: PredefinedAcl;
projection?: 'full' | 'noAcl';
userProject?: string;
}
export interface UploadConfig extends Pick<WritableOptions, 'highWaterMark'> {
/**
* The API endpoint used for the request.
* Defaults to `storage.googleapis.com`.
*
* **Warning**:
* If this value does not match the current GCP universe an emulator context
* will be assumed and authentication will be bypassed.
*/
apiEndpoint?: string;
/**
* The name of the destination bucket.
*/
bucket: string;
/**
* The name of the destination file.
*/
file: string;
/**
* The GoogleAuthOptions passed to google-auth-library
*/
authConfig?: GoogleAuthOptions;
/**
* If you want to re-use an auth client from google-auto-auth, pass an
* instance here.
* Defaults to GoogleAuth and gets automatically overridden if an
* emulator context is detected.
*/
authClient?: {
request: <T>(opts: GaxiosOptions) => Promise<GaxiosResponse<T>> | GaxiosPromise<T>;
};
/**
* Create a separate request per chunk.
*
* This value is in bytes and should be a multiple of 256 KiB (2^18).
* We recommend using at least 8 MiB for the chunk size.
*
* @link https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
*/
chunkSize?: number;
/**
* For each API request we send, you may specify custom request options that
* we'll add onto the request. The request options follow the gaxios API:
* https://github.com/googleapis/gaxios#request-options.
*/
customRequestOptions?: GaxiosOptions;
/**
* This will cause the upload to fail if the current generation of the remote
* object does not match the one provided here.
*/
generation?: number;
/**
* Set to `true` if the upload is only a subset of the overall object to upload.
* This can be used when planning to continue the upload an object in another
* session.
*
* **Must be used with {@link UploadConfig.chunkSize} != `0`**.
*
* If this is a continuation of a previous upload, {@link UploadConfig.offset}
* should be set.
*
* @see {@link checkUploadStatus} for checking the status of an existing upload.
*/
isPartialUpload?: boolean;
/**
* A customer-supplied encryption key. See
* https://cloud.google.com/storage/docs/encryption#customer-supplied.
*/
key?: string | Buffer;
/**
* Resource name of the Cloud KMS key, of the form
* `projects/my-project/locations/global/keyRings/my-kr/cryptoKeys/my-key`,
* that will be used to encrypt the object. Overrides the object metadata's
* `kms_key_name` value, if any.
*/
kmsKeyName?: string;
/**
* Any metadata you wish to set on the object.
*/
metadata?: ConfigMetadata;
/**
* The starting byte in relation to the final uploaded object.
* **Must be used with {@link UploadConfig.uri}**.
*
* If resuming an interrupted stream, do not supply this argument unless you
* know the exact number of bytes the service has AND the provided stream's
* first byte is a continuation from that provided offset. If resuming an
* interrupted stream and this option has not been provided, we will treat
* the provided upload stream as the object to upload - where the first byte
* of the upload stream is the first byte of the object to upload; skipping
* any bytes that are already present on the server.
*
* @see {@link checkUploadStatus} for checking the status of an existing upload.
* @see {@link https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload#resume-upload.}
*/
offset?: number;
/**
* Set an Origin header when creating the resumable upload URI.
*/
origin?: string;
/**
* Specify query parameters that go along with the initial upload request. See
* https://cloud.google.com/storage/docs/json_api/v1/objects/insert#parameters
*/
params?: QueryParameters;
/**
* Apply a predefined set of access controls to the created file.
*/
predefinedAcl?: PredefinedAcl;
/**
* Make the uploaded file private. (Alias for config.predefinedAcl =
* 'private')
*/
private?: boolean;
/**
* Make the uploaded file public. (Alias for config.predefinedAcl =
* 'publicRead')
*/
public?: boolean;
/**
* The service domain for a given Cloud universe.
*/
universeDomain?: string;
/**
* If you already have a resumable URI from a previously-created resumable
* upload, just pass it in here and we'll use that.
*
* If resuming an interrupted stream and the {@link UploadConfig.offset}
* option has not been provided, we will treat the provided upload stream as
* the object to upload - where the first byte of the upload stream is the
* first byte of the object to upload; skipping any bytes that are already
* present on the server.
*
* @see {@link checkUploadStatus} for checking the status of an existing upload.
*/
uri?: string;
/**
* If the bucket being accessed has requesterPays functionality enabled, this
* can be set to control which project is billed for the access of this file.
*/
userProject?: string;
/**
* Configuration options for retrying retryable errors.
*/
retryOptions: RetryOptions;
[GCCL_GCS_CMD_KEY]?: string;
}
export interface ConfigMetadata {
[key: string]: any;
/**
* Set the length of the object being uploaded. If uploading a partial
* object, this is the overall size of the finalized object.
*/
contentLength?: number;
/**
* Set the content type of the incoming data.
*/
contentType?: string;
}
export interface GoogleInnerError {
reason?: string;
}
export interface ApiError extends Error {
code?: number;
errors?: GoogleInnerError[];
}
export interface CheckUploadStatusConfig {
/**
* Set to `false` to disable retries within this method.
*
* @defaultValue `true`
*/
retry?: boolean;
}
export declare class Upload extends Writable {
#private;
bucket: string;
file: string;
apiEndpoint: string;
baseURI: string;
authConfig?: {
scopes?: string[];
};
authClient: {
request: <T>(opts: GaxiosOptions) => Promise<GaxiosResponse<T>> | GaxiosPromise<T>;
};
cacheKey: string;
chunkSize?: number;
customRequestOptions: GaxiosOptions;
generation?: number;
key?: string | Buffer;
kmsKeyName?: string;
metadata: ConfigMetadata;
offset?: number;
origin?: string;
params: QueryParameters;
predefinedAcl?: PredefinedAcl;
private?: boolean;
public?: boolean;
uri?: string;
userProject?: string;
encryption?: Encryption;
uriProvidedManually: boolean;
numBytesWritten: number;
numRetries: number;
contentLength: number | '*';
retryOptions: RetryOptions;
timeOfFirstRequest: number;
isPartialUpload: boolean;
private currentInvocationId;
/**
* A cache of buffers written to this instance, ready for consuming
*/
private writeBuffers;
private numChunksReadInRequest;
/**
* An array of buffers used for caching the most recent upload chunk.
* We should not assume that the server received all bytes sent in the request.
* - https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
*/
private localWriteCache;
private localWriteCacheByteLength;
private upstreamEnded;
constructor(cfg: UploadConfig);
/**
* Prevent 'finish' event until the upload has succeeded.
*
* @param fireFinishEvent The finish callback
*/
_final(fireFinishEvent?: () => void): void;
/**
* Handles incoming data from upstream
*
* @param chunk The chunk to append to the buffer
* @param encoding The encoding of the chunk
* @param readCallback A callback for when the buffer has been read downstream
*/
_write(chunk: Buffer | string, encoding: BufferEncoding, readCallback?: () => void): void;
/**
* Prepends the local buffer to write buffer and resets it.
*
* @param keepLastBytes number of bytes to keep from the end of the local buffer.
*/
private prependLocalBufferToUpstream;
/**
* Retrieves data from upstream's buffer.
*
* @param limit The maximum amount to return from the buffer.
*/
private pullFromChunkBuffer;
/**
* A handler for determining if data is ready to be read from upstream.
*
* @returns If there will be more chunks to read in the future
*/
private waitForNextChunk;
/**
* Reads data from upstream up to the provided `limit`.
* Ends when the limit has reached or no data is expected to be pushed from upstream.
*
* @param limit The most amount of data this iterator should return. `Infinity` by default.
*/
private upstreamIterator;
createURI(): Promise<string>;
createURI(callback: CreateUriCallback): void;
protected createURIAsync(): Promise<string>;
private continueUploading;
startUploading(): Promise<void>;
private responseHandler;
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
checkUploadStatus(config?: CheckUploadStatusConfig): Promise<GaxiosResponse<FileMetadata | void>>;
private getAndSetOffset;
private makeRequest;
private makeRequestStream;
/**
* @return {bool} is the request good?
*/
private onResponse;
/**
* @param resp GaxiosResponse object from previous attempt
*/
private attemptDelayedRetry;
/**
* The amount of time to wait before retrying the request, in milliseconds.
* If negative, do not retry.
*
* @returns the amount of time to wait, in milliseconds.
*/
private getRetryDelay;
private sanitizeEndpoint;
/**
* Check if a given status code is 2xx
*
* @param status The status code to check
* @returns if the status is 2xx
*/
isSuccessfulResponse(status: number): boolean;
}
export declare function upload(cfg: UploadConfig): Upload;
export declare function createURI(cfg: UploadConfig): Promise<string>;
export declare function createURI(cfg: UploadConfig, callback: CreateUriCallback): void;
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
export declare function checkUploadStatus(cfg: UploadConfig & Required<Pick<UploadConfig, 'uri'>>): Promise<GaxiosResponse<void | FileMetadata>>;

View File

@@ -0,0 +1,906 @@
"use strict";
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
var _Upload_instances, _Upload_gcclGcsCmd, _Upload_resetLocalBuffersCache, _Upload_addLocalBufferCache;
Object.defineProperty(exports, "__esModule", { value: true });
exports.Upload = exports.PROTOCOL_REGEX = void 0;
exports.upload = upload;
exports.createURI = createURI;
exports.checkUploadStatus = checkUploadStatus;
const abort_controller_1 = __importDefault(require("abort-controller"));
const crypto_1 = require("crypto");
const gaxios = __importStar(require("gaxios"));
const google_auth_library_1 = require("google-auth-library");
const stream_1 = require("stream");
const async_retry_1 = __importDefault(require("async-retry"));
const uuid = __importStar(require("uuid"));
const util_js_1 = require("./util.js");
const util_js_2 = require("./nodejs-common/util.js");
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
const package_json_helper_cjs_1 = require("./package-json-helper.cjs");
const NOT_FOUND_STATUS_CODE = 404;
const RESUMABLE_INCOMPLETE_STATUS_CODE = 308;
const packageJson = (0, package_json_helper_cjs_1.getPackageJSON)();
exports.PROTOCOL_REGEX = /^(\w*):\/\//;
class Upload extends stream_1.Writable {
constructor(cfg) {
var _a;
super(cfg);
_Upload_instances.add(this);
this.numBytesWritten = 0;
this.numRetries = 0;
this.currentInvocationId = {
checkUploadStatus: uuid.v4(),
chunk: uuid.v4(),
uri: uuid.v4(),
};
/**
* A cache of buffers written to this instance, ready for consuming
*/
this.writeBuffers = [];
this.numChunksReadInRequest = 0;
/**
* An array of buffers used for caching the most recent upload chunk.
* We should not assume that the server received all bytes sent in the request.
* - https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
*/
this.localWriteCache = [];
this.localWriteCacheByteLength = 0;
this.upstreamEnded = false;
_Upload_gcclGcsCmd.set(this, void 0);
cfg = cfg || {};
if (!cfg.bucket || !cfg.file) {
throw new Error('A bucket and file name are required');
}
if (cfg.offset && !cfg.uri) {
throw new RangeError('Cannot provide an `offset` without providing a `uri`');
}
if (cfg.isPartialUpload && !cfg.chunkSize) {
throw new RangeError('Cannot set `isPartialUpload` without providing a `chunkSize`');
}
cfg.authConfig = cfg.authConfig || {};
cfg.authConfig.scopes = [
'https://www.googleapis.com/auth/devstorage.full_control',
];
this.authClient = cfg.authClient || new google_auth_library_1.GoogleAuth(cfg.authConfig);
const universe = cfg.universeDomain || google_auth_library_1.DEFAULT_UNIVERSE;
this.apiEndpoint = `https://storage.${universe}`;
if (cfg.apiEndpoint && cfg.apiEndpoint !== this.apiEndpoint) {
this.apiEndpoint = this.sanitizeEndpoint(cfg.apiEndpoint);
const hostname = new URL(this.apiEndpoint).hostname;
// check if it is a domain of a known universe
const isDomain = hostname === universe;
const isDefaultUniverseDomain = hostname === google_auth_library_1.DEFAULT_UNIVERSE;
// check if it is a subdomain of a known universe
// by checking a last (universe's length + 1) of a hostname
const isSubDomainOfUniverse = hostname.slice(-(universe.length + 1)) === `.${universe}`;
const isSubDomainOfDefaultUniverse = hostname.slice(-(google_auth_library_1.DEFAULT_UNIVERSE.length + 1)) ===
`.${google_auth_library_1.DEFAULT_UNIVERSE}`;
if (!isDomain &&
!isDefaultUniverseDomain &&
!isSubDomainOfUniverse &&
!isSubDomainOfDefaultUniverse) {
// a custom, non-universe domain,
// use gaxios
this.authClient = gaxios;
}
}
this.baseURI = `${this.apiEndpoint}/upload/storage/v1/b`;
this.bucket = cfg.bucket;
const cacheKeyElements = [cfg.bucket, cfg.file];
if (typeof cfg.generation === 'number') {
cacheKeyElements.push(`${cfg.generation}`);
}
this.cacheKey = cacheKeyElements.join('/');
this.customRequestOptions = cfg.customRequestOptions || {};
this.file = cfg.file;
this.generation = cfg.generation;
this.kmsKeyName = cfg.kmsKeyName;
this.metadata = cfg.metadata || {};
this.offset = cfg.offset;
this.origin = cfg.origin;
this.params = cfg.params || {};
this.userProject = cfg.userProject;
this.chunkSize = cfg.chunkSize;
this.retryOptions = cfg.retryOptions;
this.isPartialUpload = (_a = cfg.isPartialUpload) !== null && _a !== void 0 ? _a : false;
if (cfg.key) {
if (typeof cfg.key === 'string') {
const base64Key = Buffer.from(cfg.key).toString('base64');
this.encryption = {
key: base64Key,
hash: (0, crypto_1.createHash)('sha256').update(cfg.key).digest('base64'),
};
}
else {
const base64Key = cfg.key.toString('base64');
this.encryption = {
key: base64Key,
hash: (0, crypto_1.createHash)('sha256').update(cfg.key).digest('base64'),
};
}
}
this.predefinedAcl = cfg.predefinedAcl;
if (cfg.private)
this.predefinedAcl = 'private';
if (cfg.public)
this.predefinedAcl = 'publicRead';
const autoRetry = cfg.retryOptions.autoRetry;
this.uriProvidedManually = !!cfg.uri;
this.uri = cfg.uri;
if (this.offset) {
// we're resuming an incomplete upload
this.numBytesWritten = this.offset;
}
this.numRetries = 0; // counter for number of retries currently executed
if (!autoRetry) {
cfg.retryOptions.maxRetries = 0;
}
this.timeOfFirstRequest = Date.now();
const contentLength = cfg.metadata
? Number(cfg.metadata.contentLength)
: NaN;
this.contentLength = isNaN(contentLength) ? '*' : contentLength;
__classPrivateFieldSet(this, _Upload_gcclGcsCmd, cfg[util_js_2.GCCL_GCS_CMD_KEY], "f");
this.once('writing', () => {
if (this.uri) {
this.continueUploading();
}
else {
this.createURI(err => {
if (err) {
return this.destroy(err);
}
this.startUploading();
return;
});
}
});
}
/**
* Prevent 'finish' event until the upload has succeeded.
*
* @param fireFinishEvent The finish callback
*/
_final(fireFinishEvent = () => { }) {
this.upstreamEnded = true;
this.once('uploadFinished', fireFinishEvent);
process.nextTick(() => {
this.emit('upstreamFinished');
// it's possible `_write` may not be called - namely for empty object uploads
this.emit('writing');
});
}
/**
* Handles incoming data from upstream
*
* @param chunk The chunk to append to the buffer
* @param encoding The encoding of the chunk
* @param readCallback A callback for when the buffer has been read downstream
*/
_write(chunk, encoding, readCallback = () => { }) {
// Backwards-compatible event
this.emit('writing');
this.writeBuffers.push(typeof chunk === 'string' ? Buffer.from(chunk, encoding) : chunk);
this.once('readFromChunkBuffer', readCallback);
process.nextTick(() => this.emit('wroteToChunkBuffer'));
}
/**
* Prepends the local buffer to write buffer and resets it.
*
* @param keepLastBytes number of bytes to keep from the end of the local buffer.
*/
prependLocalBufferToUpstream(keepLastBytes) {
// Typically, the upstream write buffers should be smaller than the local
// cache, so we can save time by setting the local cache as the new
// upstream write buffer array and appending the old array to it
let initialBuffers = [];
if (keepLastBytes) {
// we only want the last X bytes
let bytesKept = 0;
while (keepLastBytes > bytesKept) {
// load backwards because we want the last X bytes
// note: `localWriteCacheByteLength` is reset below
let buf = this.localWriteCache.pop();
if (!buf)
break;
bytesKept += buf.byteLength;
if (bytesKept > keepLastBytes) {
// we have gone over the amount desired, let's keep the last X bytes
// of this buffer
const diff = bytesKept - keepLastBytes;
buf = buf.subarray(diff);
bytesKept -= diff;
}
initialBuffers.unshift(buf);
}
}
else {
// we're keeping all of the local cache, simply use it as the initial buffer
initialBuffers = this.localWriteCache;
}
// Append the old upstream to the new
const append = this.writeBuffers;
this.writeBuffers = initialBuffers;
for (const buf of append) {
this.writeBuffers.push(buf);
}
// reset last buffers sent
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
}
/**
* Retrieves data from upstream's buffer.
*
* @param limit The maximum amount to return from the buffer.
*/
*pullFromChunkBuffer(limit) {
while (limit) {
const buf = this.writeBuffers.shift();
if (!buf)
break;
let bufToYield = buf;
if (buf.byteLength > limit) {
bufToYield = buf.subarray(0, limit);
this.writeBuffers.unshift(buf.subarray(limit));
limit = 0;
}
else {
limit -= buf.byteLength;
}
yield bufToYield;
// Notify upstream we've read from the buffer and we're able to consume
// more. It can also potentially send more data down as we're currently
// iterating.
this.emit('readFromChunkBuffer');
}
}
/**
* A handler for determining if data is ready to be read from upstream.
*
* @returns If there will be more chunks to read in the future
*/
async waitForNextChunk() {
const willBeMoreChunks = await new Promise(resolve => {
// There's data available - it should be digested
if (this.writeBuffers.length) {
return resolve(true);
}
// The upstream writable ended, we shouldn't expect any more data.
if (this.upstreamEnded) {
return resolve(false);
}
// Nothing immediate seems to be determined. We need to prepare some
// listeners to determine next steps...
const wroteToChunkBufferCallback = () => {
removeListeners();
return resolve(true);
};
const upstreamFinishedCallback = () => {
removeListeners();
// this should be the last chunk, if there's anything there
if (this.writeBuffers.length)
return resolve(true);
return resolve(false);
};
// Remove listeners when we're ready to callback.
const removeListeners = () => {
this.removeListener('wroteToChunkBuffer', wroteToChunkBufferCallback);
this.removeListener('upstreamFinished', upstreamFinishedCallback);
};
// If there's data recently written it should be digested
this.once('wroteToChunkBuffer', wroteToChunkBufferCallback);
// If the upstream finishes let's see if there's anything to grab
this.once('upstreamFinished', upstreamFinishedCallback);
});
return willBeMoreChunks;
}
/**
* Reads data from upstream up to the provided `limit`.
* Ends when the limit has reached or no data is expected to be pushed from upstream.
*
* @param limit The most amount of data this iterator should return. `Infinity` by default.
*/
async *upstreamIterator(limit = Infinity) {
// read from upstream chunk buffer
while (limit && (await this.waitForNextChunk())) {
// read until end or limit has been reached
for (const chunk of this.pullFromChunkBuffer(limit)) {
limit -= chunk.byteLength;
yield chunk;
}
}
}
createURI(callback) {
if (!callback) {
return this.createURIAsync();
}
this.createURIAsync().then(r => callback(null, r), callback);
}
async createURIAsync() {
const metadata = { ...this.metadata };
const headers = {};
// Delete content length and content type from metadata if they exist.
// These are headers and should not be sent as part of the metadata.
if (metadata.contentLength) {
headers['X-Upload-Content-Length'] = metadata.contentLength.toString();
delete metadata.contentLength;
}
if (metadata.contentType) {
headers['X-Upload-Content-Type'] = metadata.contentType;
delete metadata.contentType;
}
let googAPIClient = `${(0, util_js_1.getRuntimeTrackingString)()} gccl/${packageJson.version}-${(0, util_js_1.getModuleFormat)()} gccl-invocation-id/${this.currentInvocationId.uri}`;
if (__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")) {
googAPIClient += ` gccl-gcs-cmd/${__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")}`;
}
// Check if headers already exist before creating new ones
const reqOpts = {
method: 'POST',
url: [this.baseURI, this.bucket, 'o'].join('/'),
params: Object.assign({
name: this.file,
uploadType: 'resumable',
}, this.params),
data: metadata,
headers: {
'User-Agent': (0, util_js_1.getUserAgentString)(),
'x-goog-api-client': googAPIClient,
...headers,
},
};
if (metadata.contentLength) {
reqOpts.headers['X-Upload-Content-Length'] =
metadata.contentLength.toString();
}
if (metadata.contentType) {
reqOpts.headers['X-Upload-Content-Type'] = metadata.contentType;
}
if (typeof this.generation !== 'undefined') {
reqOpts.params.ifGenerationMatch = this.generation;
}
if (this.kmsKeyName) {
reqOpts.params.kmsKeyName = this.kmsKeyName;
}
if (this.predefinedAcl) {
reqOpts.params.predefinedAcl = this.predefinedAcl;
}
if (this.origin) {
reqOpts.headers.Origin = this.origin;
}
const uri = await (0, async_retry_1.default)(async (bail) => {
var _a, _b, _c;
try {
const res = await this.makeRequest(reqOpts);
// We have successfully got a URI we can now create a new invocation id
this.currentInvocationId.uri = uuid.v4();
return res.headers.location;
}
catch (err) {
const e = err;
const apiError = {
code: (_a = e.response) === null || _a === void 0 ? void 0 : _a.status,
name: (_b = e.response) === null || _b === void 0 ? void 0 : _b.statusText,
message: (_c = e.response) === null || _c === void 0 ? void 0 : _c.statusText,
errors: [
{
reason: e.code,
},
],
};
if (this.retryOptions.maxRetries > 0 &&
this.retryOptions.retryableErrorFn(apiError)) {
throw e;
}
else {
return bail(e);
}
}
}, {
retries: this.retryOptions.maxRetries,
factor: this.retryOptions.retryDelayMultiplier,
maxTimeout: this.retryOptions.maxRetryDelay * 1000, //convert to milliseconds
maxRetryTime: this.retryOptions.totalTimeout * 1000, //convert to milliseconds
});
this.uri = uri;
this.offset = 0;
// emit the newly generated URI for future reuse, if necessary.
this.emit('uri', uri);
return uri;
}
async continueUploading() {
var _a;
(_a = this.offset) !== null && _a !== void 0 ? _a : (await this.getAndSetOffset());
return this.startUploading();
}
async startUploading() {
const multiChunkMode = !!this.chunkSize;
let responseReceived = false;
this.numChunksReadInRequest = 0;
if (!this.offset) {
this.offset = 0;
}
// Check if the offset (server) is too far behind the current stream
if (this.offset < this.numBytesWritten) {
const delta = this.numBytesWritten - this.offset;
const message = `The offset is lower than the number of bytes written. The server has ${this.offset} bytes and while ${this.numBytesWritten} bytes has been uploaded - thus ${delta} bytes are missing. Stopping as this could result in data loss. Initiate a new upload to continue.`;
this.emit('error', new RangeError(message));
return;
}
// Check if we should 'fast-forward' to the relevant data to upload
if (this.numBytesWritten < this.offset) {
// 'fast-forward' to the byte where we need to upload.
// only push data from the byte after the one we left off on
const fastForwardBytes = this.offset - this.numBytesWritten;
for await (const _chunk of this.upstreamIterator(fastForwardBytes)) {
_chunk; // discard the data up until the point we want
}
this.numBytesWritten = this.offset;
}
let expectedUploadSize = undefined;
// Set `expectedUploadSize` to `contentLength - this.numBytesWritten`, if available
if (typeof this.contentLength === 'number') {
expectedUploadSize = this.contentLength - this.numBytesWritten;
}
// `expectedUploadSize` should be no more than the `chunkSize`.
// It's possible this is the last chunk request for a multiple
// chunk upload, thus smaller than the chunk size.
if (this.chunkSize) {
expectedUploadSize = expectedUploadSize
? Math.min(this.chunkSize, expectedUploadSize)
: this.chunkSize;
}
// A queue for the upstream data
const upstreamQueue = this.upstreamIterator(expectedUploadSize);
// The primary read stream for this request. This stream retrieves no more
// than the exact requested amount from upstream.
const requestStream = new stream_1.Readable({
read: async () => {
// Don't attempt to retrieve data upstream if we already have a response
if (responseReceived)
requestStream.push(null);
const result = await upstreamQueue.next();
if (result.value) {
this.numChunksReadInRequest++;
if (multiChunkMode) {
// save ever buffer used in the request in multi-chunk mode
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_addLocalBufferCache).call(this, result.value);
}
else {
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_addLocalBufferCache).call(this, result.value);
}
this.numBytesWritten += result.value.byteLength;
this.emit('progress', {
bytesWritten: this.numBytesWritten,
contentLength: this.contentLength,
});
requestStream.push(result.value);
}
if (result.done) {
requestStream.push(null);
}
},
});
let googAPIClient = `${(0, util_js_1.getRuntimeTrackingString)()} gccl/${packageJson.version}-${(0, util_js_1.getModuleFormat)()} gccl-invocation-id/${this.currentInvocationId.chunk}`;
if (__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")) {
googAPIClient += ` gccl-gcs-cmd/${__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")}`;
}
const headers = {
'User-Agent': (0, util_js_1.getUserAgentString)(),
'x-goog-api-client': googAPIClient,
};
// If using multiple chunk upload, set appropriate header
if (multiChunkMode) {
// We need to know how much data is available upstream to set the `Content-Range` header.
// https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
for await (const chunk of this.upstreamIterator(expectedUploadSize)) {
// This will conveniently track and keep the size of the buffers.
// We will reach either the expected upload size or the remainder of the stream.
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_addLocalBufferCache).call(this, chunk);
}
// This is the sum from the `#addLocalBufferCache` calls
const bytesToUpload = this.localWriteCacheByteLength;
// Important: we want to know if the upstream has ended and the queue is empty before
// unshifting data back into the queue. This way we will know if this is the last request or not.
const isLastChunkOfUpload = !(await this.waitForNextChunk());
// Important: put the data back in the queue for the actual upload
this.prependLocalBufferToUpstream();
let totalObjectSize = this.contentLength;
if (typeof this.contentLength !== 'number' &&
isLastChunkOfUpload &&
!this.isPartialUpload) {
// Let's let the server know this is the last chunk of the object since we didn't set it before.
totalObjectSize = bytesToUpload + this.numBytesWritten;
}
// `- 1` as the ending byte is inclusive in the request.
const endingByte = bytesToUpload + this.numBytesWritten - 1;
// `Content-Length` for multiple chunk uploads is the size of the chunk,
// not the overall object
headers['Content-Length'] = bytesToUpload;
headers['Content-Range'] =
`bytes ${this.offset}-${endingByte}/${totalObjectSize}`;
}
else {
headers['Content-Range'] = `bytes ${this.offset}-*/${this.contentLength}`;
}
const reqOpts = {
method: 'PUT',
url: this.uri,
headers,
body: requestStream,
};
try {
const resp = await this.makeRequestStream(reqOpts);
if (resp) {
responseReceived = true;
await this.responseHandler(resp);
}
}
catch (e) {
const err = e;
if (this.retryOptions.retryableErrorFn(err)) {
this.attemptDelayedRetry({
status: NaN,
data: err,
});
return;
}
this.destroy(err);
}
}
// Process the API response to look for errors that came in
// the response body.
async responseHandler(resp) {
if (resp.data.error) {
this.destroy(resp.data.error);
return;
}
// At this point we can safely create a new id for the chunk
this.currentInvocationId.chunk = uuid.v4();
const moreDataToUpload = await this.waitForNextChunk();
const shouldContinueWithNextMultiChunkRequest = this.chunkSize &&
resp.status === RESUMABLE_INCOMPLETE_STATUS_CODE &&
resp.headers.range &&
moreDataToUpload;
/**
* This is true when we're expecting to upload more data in a future request,
* yet the upstream for the upload session has been exhausted.
*/
const shouldContinueUploadInAnotherRequest = this.isPartialUpload &&
resp.status === RESUMABLE_INCOMPLETE_STATUS_CODE &&
!moreDataToUpload;
if (shouldContinueWithNextMultiChunkRequest) {
// Use the upper value in this header to determine where to start the next chunk.
// We should not assume that the server received all bytes sent in the request.
// https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
const range = resp.headers.range;
this.offset = Number(range.split('-')[1]) + 1;
// We should not assume that the server received all bytes sent in the request.
// - https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
const missingBytes = this.numBytesWritten - this.offset;
if (missingBytes) {
// As multi-chunk uploads send one chunk per request and pulls one
// chunk into the pipeline, prepending the missing bytes back should
// be fine for the next request.
this.prependLocalBufferToUpstream(missingBytes);
this.numBytesWritten -= missingBytes;
}
else {
// No bytes missing - no need to keep the local cache
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
}
// continue uploading next chunk
this.continueUploading();
}
else if (!this.isSuccessfulResponse(resp.status) &&
!shouldContinueUploadInAnotherRequest) {
const err = new Error('Upload failed');
err.code = resp.status;
err.name = 'Upload failed';
if (resp === null || resp === void 0 ? void 0 : resp.data) {
err.errors = [resp === null || resp === void 0 ? void 0 : resp.data];
}
this.destroy(err);
}
else {
// no need to keep the cache
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
if (resp && resp.data) {
resp.data.size = Number(resp.data.size);
}
this.emit('metadata', resp.data);
// Allow the object (Upload) to continue naturally so the user's
// "finish" event fires.
this.emit('uploadFinished');
}
}
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
async checkUploadStatus(config = {}) {
let googAPIClient = `${(0, util_js_1.getRuntimeTrackingString)()} gccl/${packageJson.version}-${(0, util_js_1.getModuleFormat)()} gccl-invocation-id/${this.currentInvocationId.checkUploadStatus}`;
if (__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")) {
googAPIClient += ` gccl-gcs-cmd/${__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")}`;
}
const opts = {
method: 'PUT',
url: this.uri,
headers: {
'Content-Length': 0,
'Content-Range': 'bytes */*',
'User-Agent': (0, util_js_1.getUserAgentString)(),
'x-goog-api-client': googAPIClient,
},
};
try {
const resp = await this.makeRequest(opts);
// Successfully got the offset we can now create a new offset invocation id
this.currentInvocationId.checkUploadStatus = uuid.v4();
return resp;
}
catch (e) {
if (config.retry === false ||
!(e instanceof Error) ||
!this.retryOptions.retryableErrorFn(e)) {
throw e;
}
const retryDelay = this.getRetryDelay();
if (retryDelay <= 0) {
throw e;
}
await new Promise(res => setTimeout(res, retryDelay));
return this.checkUploadStatus(config);
}
}
async getAndSetOffset() {
try {
// we want to handle retries in this method.
const resp = await this.checkUploadStatus({ retry: false });
if (resp.status === RESUMABLE_INCOMPLETE_STATUS_CODE) {
if (typeof resp.headers.range === 'string') {
this.offset = Number(resp.headers.range.split('-')[1]) + 1;
return;
}
}
this.offset = 0;
}
catch (e) {
const err = e;
if (this.retryOptions.retryableErrorFn(err)) {
this.attemptDelayedRetry({
status: NaN,
data: err,
});
return;
}
this.destroy(err);
}
}
async makeRequest(reqOpts) {
if (this.encryption) {
reqOpts.headers = reqOpts.headers || {};
reqOpts.headers['x-goog-encryption-algorithm'] = 'AES256';
reqOpts.headers['x-goog-encryption-key'] = this.encryption.key.toString();
reqOpts.headers['x-goog-encryption-key-sha256'] =
this.encryption.hash.toString();
}
if (this.userProject) {
reqOpts.params = reqOpts.params || {};
reqOpts.params.userProject = this.userProject;
}
// Let gaxios know we will handle a 308 error code ourselves.
reqOpts.validateStatus = (status) => {
return (this.isSuccessfulResponse(status) ||
status === RESUMABLE_INCOMPLETE_STATUS_CODE);
};
const combinedReqOpts = {
...this.customRequestOptions,
...reqOpts,
headers: {
...this.customRequestOptions.headers,
...reqOpts.headers,
},
};
const res = await this.authClient.request(combinedReqOpts);
if (res.data && res.data.error) {
throw res.data.error;
}
return res;
}
async makeRequestStream(reqOpts) {
const controller = new abort_controller_1.default();
const errorCallback = () => controller.abort();
this.once('error', errorCallback);
if (this.userProject) {
reqOpts.params = reqOpts.params || {};
reqOpts.params.userProject = this.userProject;
}
reqOpts.signal = controller.signal;
reqOpts.validateStatus = () => true;
const combinedReqOpts = {
...this.customRequestOptions,
...reqOpts,
headers: {
...this.customRequestOptions.headers,
...reqOpts.headers,
},
};
const res = await this.authClient.request(combinedReqOpts);
const successfulRequest = this.onResponse(res);
this.removeListener('error', errorCallback);
return successfulRequest ? res : null;
}
/**
* @return {bool} is the request good?
*/
onResponse(resp) {
if (resp.status !== 200 &&
this.retryOptions.retryableErrorFn({
code: resp.status,
message: resp.statusText,
name: resp.statusText,
})) {
this.attemptDelayedRetry(resp);
return false;
}
this.emit('response', resp);
return true;
}
/**
* @param resp GaxiosResponse object from previous attempt
*/
attemptDelayedRetry(resp) {
if (this.numRetries < this.retryOptions.maxRetries) {
if (resp.status === NOT_FOUND_STATUS_CODE &&
this.numChunksReadInRequest === 0) {
this.startUploading();
}
else {
const retryDelay = this.getRetryDelay();
if (retryDelay <= 0) {
this.destroy(new Error(`Retry total time limit exceeded - ${JSON.stringify(resp.data)}`));
return;
}
// Unshift the local cache back in case it's needed for the next request.
this.numBytesWritten -= this.localWriteCacheByteLength;
this.prependLocalBufferToUpstream();
// We don't know how much data has been received by the server.
// `continueUploading` will recheck the offset via `getAndSetOffset`.
// If `offset` < `numberBytesReceived` then we will raise a RangeError
// as we've streamed too much data that has been missed - this should
// not be the case for multi-chunk uploads as `lastChunkSent` is the
// body of the entire request.
this.offset = undefined;
setTimeout(this.continueUploading.bind(this), retryDelay);
}
this.numRetries++;
}
else {
this.destroy(new Error(`Retry limit exceeded - ${JSON.stringify(resp.data)}`));
}
}
/**
* The amount of time to wait before retrying the request, in milliseconds.
* If negative, do not retry.
*
* @returns the amount of time to wait, in milliseconds.
*/
getRetryDelay() {
const randomMs = Math.round(Math.random() * 1000);
const waitTime = Math.pow(this.retryOptions.retryDelayMultiplier, this.numRetries) *
1000 +
randomMs;
const maxAllowableDelayMs = this.retryOptions.totalTimeout * 1000 -
(Date.now() - this.timeOfFirstRequest);
const maxRetryDelayMs = this.retryOptions.maxRetryDelay * 1000;
return Math.min(waitTime, maxRetryDelayMs, maxAllowableDelayMs);
}
/*
* Prepare user-defined API endpoint for compatibility with our API.
*/
sanitizeEndpoint(url) {
if (!exports.PROTOCOL_REGEX.test(url)) {
url = `https://${url}`;
}
return url.replace(/\/+$/, ''); // Remove trailing slashes
}
/**
* Check if a given status code is 2xx
*
* @param status The status code to check
* @returns if the status is 2xx
*/
isSuccessfulResponse(status) {
return status >= 200 && status < 300;
}
}
exports.Upload = Upload;
_Upload_gcclGcsCmd = new WeakMap(), _Upload_instances = new WeakSet(), _Upload_resetLocalBuffersCache = function _Upload_resetLocalBuffersCache() {
this.localWriteCache = [];
this.localWriteCacheByteLength = 0;
}, _Upload_addLocalBufferCache = function _Upload_addLocalBufferCache(buf) {
this.localWriteCache.push(buf);
this.localWriteCacheByteLength += buf.byteLength;
};
function upload(cfg) {
return new Upload(cfg);
}
function createURI(cfg, callback) {
const up = new Upload(cfg);
if (!callback) {
return up.createURI();
}
up.createURI().then(r => callback(null, r), callback);
}
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
function checkUploadStatus(cfg) {
const up = new Upload(cfg);
return up.checkUploadStatus();
}

View File

@@ -0,0 +1,146 @@
import * as http from 'http';
import { Storage } from './storage.js';
import { GoogleAuth } from 'google-auth-library';
type GoogleAuthLike = Pick<GoogleAuth, 'getCredentials' | 'sign'>;
/**
* @deprecated Use {@link GoogleAuth} instead
*/
export interface AuthClient {
sign(blobToSign: string): Promise<string>;
getCredentials(): Promise<{
client_email?: string;
}>;
}
export interface BucketI {
name: string;
}
export interface FileI {
name: string;
}
export interface Query {
[key: string]: string;
}
export interface GetSignedUrlConfigInternal {
expiration: number;
accessibleAt?: Date;
method: string;
extensionHeaders?: http.OutgoingHttpHeaders;
queryParams?: Query;
cname?: string;
contentMd5?: string;
contentType?: string;
bucket: string;
file?: string;
/**
* The host for the generated signed URL
*
* @example
* 'https://localhost:8080/'
*/
host?: string | URL;
/**
* An endpoint for generating the signed URL
*
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string | URL;
}
export interface SignerGetSignedUrlConfig {
method: 'GET' | 'PUT' | 'DELETE' | 'POST';
expires: string | number | Date;
accessibleAt?: string | number | Date;
virtualHostedStyle?: boolean;
version?: 'v2' | 'v4';
cname?: string;
extensionHeaders?: http.OutgoingHttpHeaders;
queryParams?: Query;
contentMd5?: string;
contentType?: string;
/**
* The host for the generated signed URL
*
* @example
* 'https://localhost:8080/'
*/
host?: string | URL;
/**
* An endpoint for generating the signed URL
*
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string | URL;
}
export type SignerGetSignedUrlResponse = string;
export type GetSignedUrlResponse = [SignerGetSignedUrlResponse];
export interface GetSignedUrlCallback {
(err: Error | null, url?: string): void;
}
export declare enum SignerExceptionMessages {
ACCESSIBLE_DATE_INVALID = "The accessible at date provided was invalid.",
EXPIRATION_BEFORE_ACCESSIBLE_DATE = "An expiration date cannot be before accessible date.",
X_GOOG_CONTENT_SHA256 = "The header X-Goog-Content-SHA256 must be a hexadecimal string."
}
/**
* @const {string}
* @deprecated - unused
*/
export declare const PATH_STYLED_HOST = "https://storage.googleapis.com";
export declare class URLSigner {
private auth;
private bucket;
private file?;
/**
* A {@link Storage} object.
*
* @privateRemarks
*
* Technically this is a required field, however it would be a breaking change to
* move it before optional properties. In the next major we should refactor the
* constructor of this class to only accept a config object.
*/
private storage;
constructor(auth: AuthClient | GoogleAuthLike, bucket: BucketI, file?: FileI | undefined,
/**
* A {@link Storage} object.
*
* @privateRemarks
*
* Technically this is a required field, however it would be a breaking change to
* move it before optional properties. In the next major we should refactor the
* constructor of this class to only accept a config object.
*/
storage?: Storage);
getSignedUrl(cfg: SignerGetSignedUrlConfig): Promise<SignerGetSignedUrlResponse>;
private getSignedUrlV2;
private getSignedUrlV4;
/**
* Create canonical headers for signing v4 url.
*
* The canonical headers for v4-signing a request demands header names are
* first lowercased, followed by sorting the header names.
* Then, construct the canonical headers part of the request:
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
* ..
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
*
* @param headers
* @private
*/
getCanonicalHeaders(headers: http.OutgoingHttpHeaders): string;
getCanonicalRequest(method: string, path: string, query: string, headers: string, signedHeaders: string, contentSha256?: string): string;
getCanonicalQueryParams(query: Query): string;
getResourcePath(cname: boolean, bucket: string, file?: string): string;
parseExpires(expires: string | number | Date, current?: Date): number;
parseAccessibleAt(accessibleAt?: string | number | Date): number;
}
/**
* Custom error type for errors related to getting signed errors and policies.
*
* @private
*/
export declare class SigningError extends Error {
name: string;
}
export {};

View File

@@ -0,0 +1,337 @@
"use strict";
// Copyright 2020 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
Object.defineProperty(exports, "__esModule", { value: true });
exports.SigningError = exports.URLSigner = exports.PATH_STYLED_HOST = exports.SignerExceptionMessages = void 0;
const crypto = __importStar(require("crypto"));
const url = __importStar(require("url"));
const storage_js_1 = require("./storage.js");
const util_js_1 = require("./util.js");
var SignerExceptionMessages;
(function (SignerExceptionMessages) {
SignerExceptionMessages["ACCESSIBLE_DATE_INVALID"] = "The accessible at date provided was invalid.";
SignerExceptionMessages["EXPIRATION_BEFORE_ACCESSIBLE_DATE"] = "An expiration date cannot be before accessible date.";
SignerExceptionMessages["X_GOOG_CONTENT_SHA256"] = "The header X-Goog-Content-SHA256 must be a hexadecimal string.";
})(SignerExceptionMessages || (exports.SignerExceptionMessages = SignerExceptionMessages = {}));
/*
* Default signing version for getSignedUrl is 'v2'.
*/
const DEFAULT_SIGNING_VERSION = 'v2';
const SEVEN_DAYS = 7 * 24 * 60 * 60;
/**
* @const {string}
* @deprecated - unused
*/
exports.PATH_STYLED_HOST = 'https://storage.googleapis.com';
class URLSigner {
constructor(auth, bucket, file,
/**
* A {@link Storage} object.
*
* @privateRemarks
*
* Technically this is a required field, however it would be a breaking change to
* move it before optional properties. In the next major we should refactor the
* constructor of this class to only accept a config object.
*/
storage = new storage_js_1.Storage()) {
this.auth = auth;
this.bucket = bucket;
this.file = file;
this.storage = storage;
}
getSignedUrl(cfg) {
const expiresInSeconds = this.parseExpires(cfg.expires);
const method = cfg.method;
const accessibleAtInSeconds = this.parseAccessibleAt(cfg.accessibleAt);
if (expiresInSeconds < accessibleAtInSeconds) {
throw new Error(SignerExceptionMessages.EXPIRATION_BEFORE_ACCESSIBLE_DATE);
}
let customHost;
// Default style is `path`.
const isVirtualHostedStyle = cfg.virtualHostedStyle || false;
if (cfg.cname) {
customHost = cfg.cname;
}
else if (isVirtualHostedStyle) {
customHost = `https://${this.bucket.name}.storage.${this.storage.universeDomain}`;
}
const secondsToMilliseconds = 1000;
const config = Object.assign({}, cfg, {
method,
expiration: expiresInSeconds,
accessibleAt: new Date(secondsToMilliseconds * accessibleAtInSeconds),
bucket: this.bucket.name,
file: this.file ? (0, util_js_1.encodeURI)(this.file.name, false) : undefined,
});
if (customHost) {
config.cname = customHost;
}
const version = cfg.version || DEFAULT_SIGNING_VERSION;
let promise;
if (version === 'v2') {
promise = this.getSignedUrlV2(config);
}
else if (version === 'v4') {
promise = this.getSignedUrlV4(config);
}
else {
throw new Error(`Invalid signed URL version: ${version}. Supported versions are 'v2' and 'v4'.`);
}
return promise.then(query => {
var _a;
query = Object.assign(query, cfg.queryParams);
const signedUrl = new url.URL(((_a = cfg.host) === null || _a === void 0 ? void 0 : _a.toString()) || config.cname || this.storage.apiEndpoint);
signedUrl.pathname = this.getResourcePath(!!config.cname, this.bucket.name, config.file);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
signedUrl.search = (0, util_js_1.qsStringify)(query);
return signedUrl.href;
});
}
getSignedUrlV2(config) {
const canonicalHeadersString = this.getCanonicalHeaders(config.extensionHeaders || {});
const resourcePath = this.getResourcePath(false, config.bucket, config.file);
const blobToSign = [
config.method,
config.contentMd5 || '',
config.contentType || '',
config.expiration,
canonicalHeadersString + resourcePath,
].join('\n');
const sign = async () => {
var _a;
const auth = this.auth;
try {
const signature = await auth.sign(blobToSign, (_a = config.signingEndpoint) === null || _a === void 0 ? void 0 : _a.toString());
const credentials = await auth.getCredentials();
return {
GoogleAccessId: credentials.client_email,
Expires: config.expiration,
Signature: signature,
};
}
catch (err) {
const error = err;
const signingErr = new SigningError(error.message);
signingErr.stack = error.stack;
throw signingErr;
}
};
return sign();
}
getSignedUrlV4(config) {
var _a;
config.accessibleAt = config.accessibleAt
? config.accessibleAt
: new Date();
const millisecondsToSeconds = 1.0 / 1000.0;
const expiresPeriodInSeconds = config.expiration - config.accessibleAt.valueOf() * millisecondsToSeconds;
// v4 limit expiration to be 7 days maximum
if (expiresPeriodInSeconds > SEVEN_DAYS) {
throw new Error(`Max allowed expiration is seven days (${SEVEN_DAYS} seconds).`);
}
const extensionHeaders = Object.assign({}, config.extensionHeaders);
const fqdn = new url.URL(((_a = config.host) === null || _a === void 0 ? void 0 : _a.toString()) || config.cname || this.storage.apiEndpoint);
extensionHeaders.host = fqdn.hostname;
if (config.contentMd5) {
extensionHeaders['content-md5'] = config.contentMd5;
}
if (config.contentType) {
extensionHeaders['content-type'] = config.contentType;
}
let contentSha256;
const sha256Header = extensionHeaders['x-goog-content-sha256'];
if (sha256Header) {
if (typeof sha256Header !== 'string' ||
!/[A-Fa-f0-9]{40}/.test(sha256Header)) {
throw new Error(SignerExceptionMessages.X_GOOG_CONTENT_SHA256);
}
contentSha256 = sha256Header;
}
const signedHeaders = Object.keys(extensionHeaders)
.map(header => header.toLowerCase())
.sort()
.join(';');
const extensionHeadersString = this.getCanonicalHeaders(extensionHeaders);
const datestamp = (0, util_js_1.formatAsUTCISO)(config.accessibleAt);
const credentialScope = `${datestamp}/auto/storage/goog4_request`;
const sign = async () => {
var _a;
const credentials = await this.auth.getCredentials();
const credential = `${credentials.client_email}/${credentialScope}`;
const dateISO = (0, util_js_1.formatAsUTCISO)(config.accessibleAt ? config.accessibleAt : new Date(), true);
const queryParams = {
'X-Goog-Algorithm': 'GOOG4-RSA-SHA256',
'X-Goog-Credential': credential,
'X-Goog-Date': dateISO,
'X-Goog-Expires': expiresPeriodInSeconds.toString(10),
'X-Goog-SignedHeaders': signedHeaders,
...(config.queryParams || {}),
};
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const canonicalQueryParams = this.getCanonicalQueryParams(queryParams);
const canonicalRequest = this.getCanonicalRequest(config.method, this.getResourcePath(!!config.cname, config.bucket, config.file), canonicalQueryParams, extensionHeadersString, signedHeaders, contentSha256);
const hash = crypto
.createHash('sha256')
.update(canonicalRequest)
.digest('hex');
const blobToSign = [
'GOOG4-RSA-SHA256',
dateISO,
credentialScope,
hash,
].join('\n');
try {
const signature = await this.auth.sign(blobToSign, (_a = config.signingEndpoint) === null || _a === void 0 ? void 0 : _a.toString());
const signatureHex = Buffer.from(signature, 'base64').toString('hex');
const signedQuery = Object.assign({}, queryParams, {
'X-Goog-Signature': signatureHex,
});
return signedQuery;
}
catch (err) {
const error = err;
const signingErr = new SigningError(error.message);
signingErr.stack = error.stack;
throw signingErr;
}
};
return sign();
}
/**
* Create canonical headers for signing v4 url.
*
* The canonical headers for v4-signing a request demands header names are
* first lowercased, followed by sorting the header names.
* Then, construct the canonical headers part of the request:
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
* ..
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
*
* @param headers
* @private
*/
getCanonicalHeaders(headers) {
// Sort headers by their lowercased names
const sortedHeaders = (0, util_js_1.objectEntries)(headers)
// Convert header names to lowercase
.map(([headerName, value]) => [
headerName.toLowerCase(),
value,
])
.sort((a, b) => a[0].localeCompare(b[0]));
return sortedHeaders
.filter(([, value]) => value !== undefined)
.map(([headerName, value]) => {
// - Convert Array (multi-valued header) into string, delimited by
// ',' (no space).
// - Trim leading and trailing spaces.
// - Convert sequential (2+) spaces into a single space
const canonicalValue = `${value}`.trim().replace(/\s{2,}/g, ' ');
return `${headerName}:${canonicalValue}\n`;
})
.join('');
}
getCanonicalRequest(method, path, query, headers, signedHeaders, contentSha256) {
return [
method,
path,
query,
headers,
signedHeaders,
contentSha256 || 'UNSIGNED-PAYLOAD',
].join('\n');
}
getCanonicalQueryParams(query) {
return (0, util_js_1.objectEntries)(query)
.map(([key, value]) => [(0, util_js_1.encodeURI)(key, true), (0, util_js_1.encodeURI)(value, true)])
.sort((a, b) => (a[0] < b[0] ? -1 : 1))
.map(([key, value]) => `${key}=${value}`)
.join('&');
}
getResourcePath(cname, bucket, file) {
if (cname) {
return '/' + (file || '');
}
else if (file) {
return `/${bucket}/${file}`;
}
else {
return `/${bucket}`;
}
}
parseExpires(expires, current = new Date()) {
const expiresInMSeconds = new Date(expires).valueOf();
if (isNaN(expiresInMSeconds)) {
throw new Error(storage_js_1.ExceptionMessages.EXPIRATION_DATE_INVALID);
}
if (expiresInMSeconds < current.valueOf()) {
throw new Error(storage_js_1.ExceptionMessages.EXPIRATION_DATE_PAST);
}
return Math.floor(expiresInMSeconds / 1000); // The API expects seconds.
}
parseAccessibleAt(accessibleAt) {
const accessibleAtInMSeconds = new Date(accessibleAt || new Date()).valueOf();
if (isNaN(accessibleAtInMSeconds)) {
throw new Error(SignerExceptionMessages.ACCESSIBLE_DATE_INVALID);
}
return Math.floor(accessibleAtInMSeconds / 1000); // The API expects seconds.
}
}
exports.URLSigner = URLSigner;
/**
* Custom error type for errors related to getting signed errors and policies.
*
* @private
*/
class SigningError extends Error {
constructor() {
super(...arguments);
this.name = 'SigningError';
}
}
exports.SigningError = SigningError;

View File

@@ -0,0 +1,729 @@
import { ApiError, Service, ServiceOptions } from './nodejs-common/index.js';
import { Readable } from 'stream';
import { Bucket, BucketMetadata } from './bucket.js';
import { Channel } from './channel.js';
import { File } from './file.js';
import { HmacKey, HmacKeyMetadata, HmacKeyOptions } from './hmacKey.js';
import { CRC32CValidatorGenerator } from './crc32c.js';
export interface GetServiceAccountOptions {
userProject?: string;
projectIdentifier?: string;
}
export interface ServiceAccount {
emailAddress?: string;
}
export type GetServiceAccountResponse = [ServiceAccount, unknown];
export interface GetServiceAccountCallback {
(err: Error | null, serviceAccount?: ServiceAccount, apiResponse?: unknown): void;
}
export interface CreateBucketQuery {
enableObjectRetention: boolean;
predefinedAcl?: 'authenticatedRead' | 'private' | 'projectPrivate' | 'publicRead' | 'publicReadWrite';
predefinedDefaultObjectAcl?: 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
project: string;
projection?: 'full' | 'noAcl';
userProject: string;
}
export declare enum IdempotencyStrategy {
RetryAlways = 0,
RetryConditional = 1,
RetryNever = 2
}
export interface RetryOptions {
retryDelayMultiplier?: number;
totalTimeout?: number;
maxRetryDelay?: number;
autoRetry?: boolean;
maxRetries?: number;
retryableErrorFn?: (err: ApiError) => boolean;
idempotencyStrategy?: IdempotencyStrategy;
}
export interface PreconditionOptions {
ifGenerationMatch?: number | string;
ifGenerationNotMatch?: number | string;
ifMetagenerationMatch?: number | string;
ifMetagenerationNotMatch?: number | string;
}
export interface StorageOptions extends ServiceOptions {
/**
* The API endpoint of the service used to make requests.
* Defaults to `storage.googleapis.com`.
*/
apiEndpoint?: string;
crc32cGenerator?: CRC32CValidatorGenerator;
retryOptions?: RetryOptions;
}
export interface BucketOptions {
crc32cGenerator?: CRC32CValidatorGenerator;
kmsKeyName?: string;
preconditionOpts?: PreconditionOptions;
userProject?: string;
generation?: number;
softDeleted?: boolean;
}
export interface Cors {
maxAgeSeconds?: number;
method?: string[];
origin?: string[];
responseHeader?: string[];
}
interface Versioning {
enabled: boolean;
}
/**
* Custom placement configuration.
* Initially used for dual-region buckets.
**/
export interface CustomPlacementConfig {
dataLocations?: string[];
}
export interface AutoclassConfig {
enabled?: boolean;
terminalStorageClass?: 'NEARLINE' | 'ARCHIVE';
}
export interface CreateBucketRequest extends BucketMetadata {
archive?: boolean;
coldline?: boolean;
dataLocations?: string[];
dra?: boolean;
enableObjectRetention?: boolean;
multiRegional?: boolean;
nearline?: boolean;
predefinedAcl?: 'authenticatedRead' | 'private' | 'projectPrivate' | 'publicRead' | 'publicReadWrite';
predefinedDefaultObjectAcl?: 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
projection?: 'full' | 'noAcl';
regional?: boolean;
requesterPays?: boolean;
rpo?: string;
standard?: boolean;
storageClass?: string;
userProject?: string;
versioning?: Versioning;
}
export type CreateBucketResponse = [Bucket, unknown];
export interface BucketCallback {
(err: Error | null, bucket?: Bucket | null, apiResponse?: unknown): void;
}
export type GetBucketsResponse = [Bucket[], {}, unknown];
export interface GetBucketsCallback {
(err: Error | null, buckets: Bucket[], nextQuery?: {}, apiResponse?: unknown): void;
}
export interface GetBucketsRequest {
prefix?: string;
project?: string;
autoPaginate?: boolean;
maxApiCalls?: number;
maxResults?: number;
pageToken?: string;
userProject?: string;
softDeleted?: boolean;
generation?: number;
}
export interface HmacKeyResourceResponse {
metadata: HmacKeyMetadata;
secret: string;
}
export type CreateHmacKeyResponse = [HmacKey, string, HmacKeyResourceResponse];
export interface CreateHmacKeyOptions {
projectId?: string;
userProject?: string;
}
export interface CreateHmacKeyCallback {
(err: Error | null, hmacKey?: HmacKey | null, secret?: string | null, apiResponse?: HmacKeyResourceResponse): void;
}
export interface GetHmacKeysOptions {
projectId?: string;
serviceAccountEmail?: string;
showDeletedKeys?: boolean;
autoPaginate?: boolean;
maxApiCalls?: number;
maxResults?: number;
pageToken?: string;
userProject?: string;
}
export interface GetHmacKeysCallback {
(err: Error | null, hmacKeys: HmacKey[] | null, nextQuery?: {}, apiResponse?: unknown): void;
}
export declare enum ExceptionMessages {
EXPIRATION_DATE_INVALID = "The expiration date provided was invalid.",
EXPIRATION_DATE_PAST = "An expiration date cannot be in the past."
}
export declare enum StorageExceptionMessages {
BUCKET_NAME_REQUIRED = "A bucket name is needed to use Cloud Storage.",
BUCKET_NAME_REQUIRED_CREATE = "A name is required to create a bucket.",
HMAC_SERVICE_ACCOUNT = "The first argument must be a service account email to create an HMAC key.",
HMAC_ACCESS_ID = "An access ID is needed to create an HmacKey object."
}
export type GetHmacKeysResponse = [HmacKey[]];
export declare const PROTOCOL_REGEX: RegExp;
/**
* Default behavior: Automatically retry retriable server errors.
*
* @const {boolean}
*/
export declare const AUTO_RETRY_DEFAULT = true;
/**
* Default behavior: Only attempt to retry retriable errors 3 times.
*
* @const {number}
*/
export declare const MAX_RETRY_DEFAULT = 3;
/**
* Default behavior: Wait twice as long as previous retry before retrying.
*
* @const {number}
*/
export declare const RETRY_DELAY_MULTIPLIER_DEFAULT = 2;
/**
* Default behavior: If the operation doesn't succeed after 600 seconds,
* stop retrying.
*
* @const {number}
*/
export declare const TOTAL_TIMEOUT_DEFAULT = 600;
/**
* Default behavior: Wait no more than 64 seconds between retries.
*
* @const {number}
*/
export declare const MAX_RETRY_DELAY_DEFAULT = 64;
/**
* Returns true if the API request should be retried, given the error that was
* given the first time the request was attempted.
* @const
* @param {error} err - The API error to check if it is appropriate to retry.
* @return {boolean} True if the API request should be retried, false otherwise.
*/
export declare const RETRYABLE_ERR_FN_DEFAULT: (err?: ApiError) => boolean;
/*! Developer Documentation
*
* Invoke this method to create a new Storage object bound with pre-determined
* configuration options. For each object that can be created (e.g., a bucket),
* there is an equivalent static and instance method. While they are classes,
* they can be instantiated without use of the `new` keyword.
*/
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* This object provides constants to refer to the three permission levels that
* can be granted to an entity:
*
* - `gcs.acl.OWNER_ROLE` - ("OWNER")
* - `gcs.acl.READER_ROLE` - ("READER")
* - `gcs.acl.WRITER_ROLE` - ("WRITER")
*
* See {@link https://cloud.google.com/storage/docs/access-control/lists| About Access Control Lists}
*
* @name Storage#acl
* @type {object}
* @property {string} OWNER_ROLE
* @property {string} READER_ROLE
* @property {string} WRITER_ROLE
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const albums = storage.bucket('albums');
*
* //-
* // Make all of the files currently in a bucket publicly readable.
* //-
* const options = {
* entity: 'allUsers',
* role: storage.acl.READER_ROLE
* };
*
* albums.acl.add(options, function(err, aclObject) {});
*
* //-
* // Make any new objects added to a bucket publicly readable.
* //-
* albums.acl.default.add(options, function(err, aclObject) {});
*
* //-
* // Grant a user ownership permissions to a bucket.
* //-
* albums.acl.add({
* entity: 'user-useremail@example.com',
* role: storage.acl.OWNER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* albums.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
/**
* Get {@link Bucket} objects for all of the buckets in your project as
* a readable object stream.
*
* @method Storage#getBucketsStream
* @param {GetBucketsRequest} [query] Query object for listing buckets.
* @returns {ReadableStream} A readable stream that emits {@link Bucket}
* instances.
*
* @example
* ```
* storage.getBucketsStream()
* .on('error', console.error)
* .on('data', function(bucket) {
* // bucket is a Bucket object.
* })
* .on('end', function() {
* // All buckets retrieved.
* });
*
* //-
* // If you anticipate many results, you can end a stream early to prevent
* // unnecessary processing and API requests.
* //-
* storage.getBucketsStream()
* .on('data', function(bucket) {
* this.end();
* });
* ```
*/
/**
* Get {@link HmacKey} objects for all of the HMAC keys in the project in a
* readable object stream.
*
* @method Storage#getHmacKeysStream
* @param {GetHmacKeysOptions} [options] Configuration options.
* @returns {ReadableStream} A readable stream that emits {@link HmacKey}
* instances.
*
* @example
* ```
* storage.getHmacKeysStream()
* .on('error', console.error)
* .on('data', function(hmacKey) {
* // hmacKey is an HmacKey object.
* })
* .on('end', function() {
* // All HmacKey retrieved.
* });
*
* //-
* // If you anticipate many results, you can end a stream early to prevent
* // unnecessary processing and API requests.
* //-
* storage.getHmacKeysStream()
* .on('data', function(bucket) {
* this.end();
* });
* ```
*/
/**
* <h4>ACLs</h4>
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share files with other users
* and allow other users to access your buckets and files.
*
* To learn more about ACLs, read this overview on
* {@link https://cloud.google.com/storage/docs/access-control| Access Control}.
*
* See {@link https://cloud.google.com/storage/docs/overview| Cloud Storage overview}
* See {@link https://cloud.google.com/storage/docs/access-control| Access Control}
*
* @class
*/
export declare class Storage extends Service {
/**
* {@link Bucket} class.
*
* @name Storage.Bucket
* @see Bucket
* @type {Constructor}
*/
static Bucket: typeof Bucket;
/**
* {@link Channel} class.
*
* @name Storage.Channel
* @see Channel
* @type {Constructor}
*/
static Channel: typeof Channel;
/**
* {@link File} class.
*
* @name Storage.File
* @see File
* @type {Constructor}
*/
static File: typeof File;
/**
* {@link HmacKey} class.
*
* @name Storage.HmacKey
* @see HmacKey
* @type {Constructor}
*/
static HmacKey: typeof HmacKey;
static acl: {
OWNER_ROLE: string;
READER_ROLE: string;
WRITER_ROLE: string;
};
/**
* Reference to {@link Storage.acl}.
*
* @name Storage#acl
* @see Storage.acl
*/
acl: typeof Storage.acl;
crc32cGenerator: CRC32CValidatorGenerator;
getBucketsStream(): Readable;
getHmacKeysStream(): Readable;
retryOptions: RetryOptions;
/**
* @callback Crc32cGeneratorToStringCallback
* A method returning the CRC32C as a base64-encoded string.
*
* @returns {string}
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
/**
* @callback Crc32cGeneratorValidateCallback
* A method validating a base64-encoded CRC32C string.
*
* @param {string} [value] base64-encoded CRC32C string to validate
* @returns {boolean}
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
**/
/**
* @callback Crc32cGeneratorUpdateCallback
* A method for passing `Buffer`s for CRC32C generation.
*
* @param {Buffer} [data] data to update CRC32C value with
* @returns {undefined}
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
**/
/**
* @typedef {object} CRC32CValidator
* @property {Crc32cGeneratorToStringCallback}
* @property {Crc32cGeneratorValidateCallback}
* @property {Crc32cGeneratorUpdateCallback}
*/
/**
* @callback Crc32cGeneratorCallback
* @returns {CRC32CValidator}
*/
/**
* @typedef {object} StorageOptions
* @property {string} [projectId] The project ID from the Google Developer's
* Console, e.g. 'grape-spaceship-123'. We will also check the environment
* variable `GCLOUD_PROJECT` for your project ID. If your app is running
* in an environment which supports {@link
* https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application
* Application Default Credentials}, your project ID will be detected
* automatically.
* @property {string} [keyFilename] Full path to the a .json, .pem, or .p12 key
* downloaded from the Google Developers Console. If you provide a path to
* a JSON file, the `projectId` option above is not necessary. NOTE: .pem and
* .p12 require you to specify the `email` option as well.
* @property {string} [email] Account email address. Required when using a .pem
* or .p12 keyFilename.
* @property {object} [credentials] Credentials object.
* @property {string} [credentials.client_email]
* @property {string} [credentials.private_key]
* @property {object} [retryOptions] Options for customizing retries. Retriable server errors
* will be retried with exponential delay between them dictated by the formula
* max(maxRetryDelay, retryDelayMultiplier*retryNumber) until maxRetries or totalTimeout
* has been reached. Retries will only happen if autoRetry is set to true.
* @property {boolean} [retryOptions.autoRetry=true] Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* @property {number} [retryOptions.retryDelayMultiplier = 2] the multiplier by which to
* increase the delay time between the completion of failed requests, and the
* initiation of the subsequent retrying request.
* @property {number} [retryOptions.totalTimeout = 600] The total time, starting from
* when the initial request is sent, after which an error will
* be returned, regardless of the retrying attempts made meanwhile.
* @property {number} [retryOptions.maxRetryDelay = 64] The maximum delay time between requests.
* When this value is reached, ``retryDelayMultiplier`` will no longer be used to
* increase delay time.
* @property {number} [retryOptions.maxRetries=3] Maximum number of automatic retries
* attempted before returning the error.
* @property {function} [retryOptions.retryableErrorFn] Function that returns true if a given
* error should be retried and false otherwise.
* @property {enum} [retryOptions.idempotencyStrategy=IdempotencyStrategy.RetryConditional] Enumeration
* controls how conditionally idempotent operations are retried. Possible values are: RetryAlways -
* will respect other retry settings and attempt to retry conditionally idempotent operations. RetryConditional -
* will retry conditionally idempotent operations if the correct preconditions are set. RetryNever - never
* retry a conditionally idempotent operation.
* @property {string} [userAgent] The value to be prepended to the User-Agent
* header in API requests.
* @property {object} [authClient] `AuthClient` or `GoogleAuth` client to reuse instead of creating a new one.
* @property {number} [timeout] The amount of time in milliseconds to wait per http request before timing out.
* @property {object[]} [interceptors_] Array of custom request interceptors to be returned in the order they were assigned.
* @property {string} [apiEndpoint = storage.google.com] The API endpoint of the service used to make requests.
* @property {boolean} [useAuthWithCustomEndpoint = false] Controls whether or not to use authentication when using a custom endpoint.
* @property {Crc32cGeneratorCallback} [callback] A function that generates a CRC32C Validator. Defaults to {@link CRC32C}
*/
/**
* Constructs the Storage client.
*
* @example
* Create a client that uses Application Default Credentials
* (ADC)
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* ```
*
* @example
* Create a client with explicit credentials
* ```
* const storage = new Storage({
* projectId: 'your-project-id',
* keyFilename: '/path/to/keyfile.json'
* });
* ```
*
* @example
* Create a client with credentials passed
* by value as a JavaScript object
* ```
* const storage = new Storage({
* projectId: 'your-project-id',
* credentials: {
* type: 'service_account',
* project_id: 'xxxxxxx',
* private_key_id: 'xxxx',
* private_key:'-----BEGIN PRIVATE KEY-----xxxxxxx\n-----END PRIVATE KEY-----\n',
* client_email: 'xxxx',
* client_id: 'xxx',
* auth_uri: 'https://accounts.google.com/o/oauth2/auth',
* token_uri: 'https://oauth2.googleapis.com/token',
* auth_provider_x509_cert_url: 'https://www.googleapis.com/oauth2/v1/certs',
* client_x509_cert_url: 'xxx',
* }
* });
* ```
*
* @example
* Create a client with credentials passed
* by loading a JSON file directly from disk
* ```
* const storage = new Storage({
* projectId: 'your-project-id',
* credentials: require('/path/to-keyfile.json')
* });
* ```
*
* @example
* Create a client with an `AuthClient` (e.g. `DownscopedClient`)
* ```
* const {DownscopedClient} = require('google-auth-library');
* const authClient = new DownscopedClient({...});
*
* const storage = new Storage({authClient});
* ```
*
* Additional samples:
* - https://github.com/googleapis/google-auth-library-nodejs#sample-usage-1
* - https://github.com/googleapis/google-auth-library-nodejs/blob/main/samples/downscopedclient.js
*
* @param {StorageOptions} [options] Configuration options.
*/
constructor(options?: StorageOptions);
private static sanitizeEndpoint;
/**
* Get a reference to a Cloud Storage bucket.
*
* @param {string} name Name of the bucket.
* @param {object} [options] Configuration object.
* @param {string} [options.kmsKeyName] A Cloud KMS key that will be used to
* encrypt objects inserted into this bucket, if no encryption method is
* specified.
* @param {string} [options.userProject] User project to be billed for all
* requests made from this Bucket object.
* @returns {Bucket}
* @see Bucket
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const albums = storage.bucket('albums');
* const photos = storage.bucket('photos');
* ```
*/
bucket(name: string, options?: BucketOptions): Bucket;
/**
* Reference a channel to receive notifications about changes to your bucket.
*
* @param {string} id The ID of the channel.
* @param {string} resourceId The resource ID of the channel.
* @returns {Channel}
* @see Channel
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* ```
*/
channel(id: string, resourceId: string): Channel;
createBucket(name: string, metadata?: CreateBucketRequest): Promise<CreateBucketResponse>;
createBucket(name: string, callback: BucketCallback): void;
createBucket(name: string, metadata: CreateBucketRequest, callback: BucketCallback): void;
createBucket(name: string, metadata: CreateBucketRequest, callback: BucketCallback): void;
createHmacKey(serviceAccountEmail: string, options?: CreateHmacKeyOptions): Promise<CreateHmacKeyResponse>;
createHmacKey(serviceAccountEmail: string, callback: CreateHmacKeyCallback): void;
createHmacKey(serviceAccountEmail: string, options: CreateHmacKeyOptions, callback: CreateHmacKeyCallback): void;
getBuckets(options?: GetBucketsRequest): Promise<GetBucketsResponse>;
getBuckets(options: GetBucketsRequest, callback: GetBucketsCallback): void;
getBuckets(callback: GetBucketsCallback): void;
/**
* Query object for listing HMAC keys.
*
* @typedef {object} GetHmacKeysOptions
* @property {string} [projectId] The project ID of the project that owns
* the service account of the requested HMAC key. If not provided,
* the project ID used to instantiate the Storage client will be used.
* @property {string} [serviceAccountEmail] If present, only HMAC keys for the
* given service account are returned.
* @property {boolean} [showDeletedKeys=false] If true, include keys in the DELETE
* state. Default is false.
* @property {boolean} [autoPaginate=true] Have pagination handled
* automatically.
* @property {number} [maxApiCalls] Maximum number of API calls to make.
* @property {number} [maxResults] Maximum number of items plus prefixes to
* return per call.
* Note: By default will handle pagination automatically
* if more than 1 page worth of results are requested per call.
* When `autoPaginate` is set to `false` the smaller of `maxResults`
* or 1 page of results will be returned per call.
* @property {string} [pageToken] A previously-returned page token
* representing part of the larger set of results to view.
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* @typedef {array} GetHmacKeysResponse
* @property {HmacKey[]} 0 Array of {@link HmacKey} instances.
* @param {object} nextQuery 1 A query object to receive more results.
* @param {object} apiResponse 2 The full API response.
*/
/**
* @callback GetHmacKeysCallback
* @param {?Error} err Request error, if any.
* @param {HmacKey[]} hmacKeys Array of {@link HmacKey} instances.
* @param {object} nextQuery A query object to receive more results.
* @param {object} apiResponse The full API response.
*/
/**
* Retrieves a list of HMAC keys matching the criteria.
*
* The authenticated user must have storage.hmacKeys.list permission for the project in which the key exists.
*
* @param {GetHmacKeysOption} options Configuration options.
* @param {GetHmacKeysCallback} callback Callback function.
* @return {Promise<GetHmacKeysResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* storage.getHmacKeys(function(err, hmacKeys) {
* if (!err) {
* // hmacKeys is an array of HmacKey objects.
* }
* });
*
* //-
* // To control how many API requests are made and page through the results
* // manually, set `autoPaginate` to `false`.
* //-
* const callback = function(err, hmacKeys, nextQuery, apiResponse) {
* if (nextQuery) {
* // More results exist.
* storage.getHmacKeys(nextQuery, callback);
* }
*
* // The `metadata` property is populated for you with the metadata at the
* // time of fetching.
* hmacKeys[0].metadata;
* };
*
* storage.getHmacKeys({
* autoPaginate: false
* }, callback);
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* storage.getHmacKeys().then(function(data) {
* const hmacKeys = data[0];
* });
* ```
*/
getHmacKeys(options?: GetHmacKeysOptions): Promise<GetHmacKeysResponse>;
getHmacKeys(callback: GetHmacKeysCallback): void;
getHmacKeys(options: GetHmacKeysOptions, callback: GetHmacKeysCallback): void;
getServiceAccount(options?: GetServiceAccountOptions): Promise<GetServiceAccountResponse>;
getServiceAccount(options?: GetServiceAccountOptions): Promise<GetServiceAccountResponse>;
getServiceAccount(options: GetServiceAccountOptions, callback: GetServiceAccountCallback): void;
getServiceAccount(callback: GetServiceAccountCallback): void;
/**
* Get a reference to an HmacKey object.
* Note: this does not fetch the HMAC key's metadata. Use HmacKey#get() to
* retrieve and populate the metadata.
*
* To get a reference to an HMAC key that's not created for a service
* account in the same project used to instantiate the Storage client,
* supply the project's ID as `projectId` in the `options` argument.
*
* @param {string} accessId The HMAC key's access ID.
* @param {HmacKeyOptions} options HmacKey constructor options.
* @returns {HmacKey}
* @see HmacKey
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const hmacKey = storage.hmacKey('ACCESS_ID');
* ```
*/
hmacKey(accessId: string, options?: HmacKeyOptions): HmacKey;
}
export {};

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,253 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { Bucket, UploadOptions, UploadResponse } from './bucket.js';
import { DownloadOptions, DownloadResponse, File } from './file.js';
import { GaxiosResponse } from 'gaxios';
export interface UploadManyFilesOptions {
concurrencyLimit?: number;
customDestinationBuilder?(path: string, options: UploadManyFilesOptions): string;
skipIfExists?: boolean;
prefix?: string;
passthroughOptions?: Omit<UploadOptions, 'destination'>;
}
export interface DownloadManyFilesOptions {
concurrencyLimit?: number;
prefix?: string;
stripPrefix?: string;
passthroughOptions?: DownloadOptions;
skipIfExists?: boolean;
}
export interface DownloadFileInChunksOptions {
concurrencyLimit?: number;
chunkSizeBytes?: number;
destination?: string;
validation?: 'crc32c' | false;
noReturnData?: boolean;
}
export interface UploadFileInChunksOptions {
concurrencyLimit?: number;
chunkSizeBytes?: number;
uploadName?: string;
maxQueueSize?: number;
uploadId?: string;
autoAbortFailure?: boolean;
partsMap?: Map<number, string>;
validation?: 'md5' | false;
headers?: {
[key: string]: string;
};
}
export interface MultiPartUploadHelper {
bucket: Bucket;
fileName: string;
uploadId?: string;
partsMap?: Map<number, string>;
initiateUpload(headers?: {
[key: string]: string;
}): Promise<void>;
uploadPart(partNumber: number, chunk: Buffer, validation?: 'md5' | false): Promise<void>;
completeUpload(): Promise<GaxiosResponse | undefined>;
abortUpload(): Promise<void>;
}
export type MultiPartHelperGenerator = (bucket: Bucket, fileName: string, uploadId?: string, partsMap?: Map<number, string>) => MultiPartUploadHelper;
export declare class MultiPartUploadError extends Error {
private uploadId;
private partsMap;
constructor(message: string, uploadId: string, partsMap: Map<number, string>);
}
/**
* Create a TransferManager object to perform parallel transfer operations on a Cloud Storage bucket.
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket A {@link Bucket} instance
*
*/
export declare class TransferManager {
bucket: Bucket;
constructor(bucket: Bucket);
/**
* @typedef {object} UploadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the files.
* @property {Function} [customDestinationBuilder] A function that will take the current path of a local file
* and return a string representing a custom path to be used to upload the file to GCS.
* @property {boolean} [skipIfExists] Do not upload the file if it already exists in
* the bucket. This will set the precondition ifGenerationMatch = 0.
* @property {string} [prefix] A prefix to append to all of the uploaded files.
* @property {object} [passthroughOptions] {@link UploadOptions} Options to be passed through
* to each individual upload operation.
*
*/
/**
* Upload multiple files in parallel to the bucket. This is a convenience method
* that utilizes {@link Bucket#upload} to perform the upload.
*
* @param {array | string} [filePathsOrDirectory] An array of fully qualified paths to the files or a directory name.
* If a directory name is provided, the directory will be recursively walked and all files will be added to the upload list.
* to be uploaded to the bucket
* @param {UploadManyFilesOptions} [options] Configuration options.
* @returns {Promise<UploadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload multiple files in parallel.
* //-
* const response = await transferManager.uploadManyFiles(['/local/path/file1.txt, 'local/path/file2.txt']);
* // Your bucket now contains:
* // - "local/path/file1.txt" (with the contents of '/local/path/file1.txt')
* // - "local/path/file2.txt" (with the contents of '/local/path/file2.txt')
* const response = await transferManager.uploadManyFiles('/local/directory');
* // Your bucket will now contain all files contained in '/local/directory' maintaining the subdirectory structure.
* ```
*
*/
uploadManyFiles(filePathsOrDirectory: string[] | string, options?: UploadManyFilesOptions): Promise<UploadResponse[]>;
/**
* @typedef {object} DownloadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the files.
* @property {string} [prefix] A prefix to append to all of the downloaded files.
* @property {string} [stripPrefix] A prefix to remove from all of the downloaded files.
* @property {object} [passthroughOptions] {@link DownloadOptions} Options to be passed through
* to each individual download operation.
* @property {boolean} [skipIfExists] Do not download the file if it already exists in
* the destination.
*
*/
/**
* Download multiple files in parallel to the local filesystem. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {array | string} [filesOrFolder] An array of file name strings or file objects to be downloaded. If
* a string is provided this will be treated as a GCS prefix and all files with that prefix will be downloaded.
* @param {DownloadManyFilesOptions} [options] Configuration options. Setting options.prefix or options.stripPrefix
* or options.passthroughOptions.destination will cause the downloaded files to be written to the file system
* instead of being returned as a buffer.
* @returns {Promise<DownloadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download multiple files in parallel.
* //-
* const response = await transferManager.downloadManyFiles(['file1.txt', 'file2.txt']);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles([bucket.File('file1.txt'), bucket.File('file2.txt')]);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles('test-folder');
* // All files with GCS prefix of 'test-folder' have been downloaded.
* ```
*
*/
downloadManyFiles(filesOrFolder: File[] | string[] | string, options?: DownloadManyFilesOptions): Promise<void | DownloadResponse[]>;
/**
* @typedef {object} DownloadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be downloaded.
* @property {string | boolean} [validation] Whether or not to perform a CRC32C validation check when download is complete.
* @property {boolean} [noReturnData] Whether or not to return the downloaded data. A `true` value here would be useful for files with a size that will not fit into memory.
*
*/
/**
* Download a large file in chunks utilizing parallel download operations. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {File | string} fileOrName {@link File} to download.
* @param {DownloadFileInChunksOptions} [options] Configuration options.
* @returns {Promise<void | DownloadResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.downloadFileInChunks(bucket.file('large-file.txt');
* // Your local directory now contains:
* // - "large-file.txt" (with the contents from my-bucket.large-file.txt)
* ```
*
*/
downloadFileInChunks(fileOrName: File | string, options?: DownloadFileInChunksOptions): Promise<void | DownloadResponse>;
/**
* @typedef {object} UploadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be uploaded.
* @property {string} [uploadName] Name of the file when saving to GCS. If omitted the name is taken from the file path.
* @property {number} [maxQueueSize] The number of chunks to be uploaded to hold in memory concurrently. If not specified
* defaults to the specified concurrency limit.
* @property {string} [uploadId] If specified attempts to resume a previous upload.
* @property {Map} [partsMap] If specified alongside uploadId, attempts to resume a previous upload from the last chunk
* specified in partsMap
* @property {object} [headers] headers to be sent when initiating the multipart upload.
* See {@link https://cloud.google.com/storage/docs/xml-api/post-object-multipart#request_headers| Request Headers: Initiate a Multipart Upload}
* @property {boolean} [autoAbortFailure] boolean to indicate if an in progress upload session will be automatically aborted upon failure. If not set,
* failures will be automatically aborted.
*
*/
/**
* Upload a large file in chunks utilizing parallel upload operations. If the upload fails, an uploadId and
* map containing all the successfully uploaded parts will be returned to the caller. These arguments can be used to
* resume the upload.
*
* @param {string} [filePath] The path of the file to be uploaded
* @param {UploadFileInChunksOptions} [options] Configuration options.
* @param {MultiPartHelperGenerator} [generator] A function that will return a type that implements the MPU interface. Most users will not need to use this.
* @returns {Promise<void>} If successful a promise resolving to void, otherwise a error containing the message, uploadId, and parts map.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.uploadFileInChunks('large-file.txt');
* // Your bucket now contains:
* // - "large-file.txt"
* ```
*
*
*/
uploadFileInChunks(filePath: string, options?: UploadFileInChunksOptions, generator?: MultiPartHelperGenerator): Promise<GaxiosResponse | undefined>;
private getPathsFromDirectory;
}

View File

@@ -0,0 +1,694 @@
"use strict";
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
var _XMLMultiPartUploadHelper_instances, _XMLMultiPartUploadHelper_setGoogApiClientHeaders, _XMLMultiPartUploadHelper_handleErrorResponse;
Object.defineProperty(exports, "__esModule", { value: true });
exports.TransferManager = exports.MultiPartUploadError = void 0;
const file_js_1 = require("./file.js");
const p_limit_1 = __importDefault(require("p-limit"));
const path = __importStar(require("path"));
const fs_1 = require("fs");
const crc32c_js_1 = require("./crc32c.js");
const google_auth_library_1 = require("google-auth-library");
const fast_xml_parser_1 = require("fast-xml-parser");
const async_retry_1 = __importDefault(require("async-retry"));
const crypto_1 = require("crypto");
const util_js_1 = require("./nodejs-common/util.js");
const util_js_2 = require("./util.js");
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
const package_json_helper_cjs_1 = require("./package-json-helper.cjs");
const packageJson = (0, package_json_helper_cjs_1.getPackageJSON)();
/**
* Default number of concurrently executing promises to use when calling uploadManyFiles.
*
*/
const DEFAULT_PARALLEL_UPLOAD_LIMIT = 5;
/**
* Default number of concurrently executing promises to use when calling downloadManyFiles.
*
*/
const DEFAULT_PARALLEL_DOWNLOAD_LIMIT = 5;
/**
* Default number of concurrently executing promises to use when calling downloadFileInChunks.
*
*/
const DEFAULT_PARALLEL_CHUNKED_DOWNLOAD_LIMIT = 5;
/**
* The minimum size threshold in bytes at which to apply a chunked download strategy when calling downloadFileInChunks.
*
*/
const DOWNLOAD_IN_CHUNKS_FILE_SIZE_THRESHOLD = 32 * 1024 * 1024;
/**
* The chunk size in bytes to use when calling downloadFileInChunks.
*
*/
const DOWNLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE = 32 * 1024 * 1024;
/**
* The chunk size in bytes to use when calling uploadFileInChunks.
*
*/
const UPLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE = 32 * 1024 * 1024;
/**
* Default number of concurrently executing promises to use when calling uploadFileInChunks.
*
*/
const DEFAULT_PARALLEL_CHUNKED_UPLOAD_LIMIT = 5;
const EMPTY_REGEX = '(?:)';
/**
* The `gccl-gcs-cmd` value for the `X-Goog-API-Client` header.
* Example: `gccl-gcs-cmd/tm.upload_many`
*
* @see {@link GCCL_GCS_CMD}.
* @see {@link GCCL_GCS_CMD_KEY}.
*/
const GCCL_GCS_CMD_FEATURE = {
UPLOAD_MANY: 'tm.upload_many',
DOWNLOAD_MANY: 'tm.download_many',
UPLOAD_SHARDED: 'tm.upload_sharded',
DOWNLOAD_SHARDED: 'tm.download_sharded',
};
const defaultMultiPartGenerator = (bucket, fileName, uploadId, partsMap) => {
return new XMLMultiPartUploadHelper(bucket, fileName, uploadId, partsMap);
};
class MultiPartUploadError extends Error {
constructor(message, uploadId, partsMap) {
super(message);
this.uploadId = uploadId;
this.partsMap = partsMap;
}
}
exports.MultiPartUploadError = MultiPartUploadError;
/**
* Class representing an implementation of MPU in the XML API. This class is not meant for public usage.
*
* @private
*
*/
class XMLMultiPartUploadHelper {
constructor(bucket, fileName, uploadId, partsMap) {
_XMLMultiPartUploadHelper_instances.add(this);
this.authClient = bucket.storage.authClient || new google_auth_library_1.GoogleAuth();
this.uploadId = uploadId || '';
this.bucket = bucket;
this.fileName = fileName;
this.baseUrl = `https://${bucket.name}.${new URL(this.bucket.storage.apiEndpoint).hostname}/${fileName}`;
this.xmlBuilder = new fast_xml_parser_1.XMLBuilder({ arrayNodeName: 'Part' });
this.xmlParser = new fast_xml_parser_1.XMLParser();
this.partsMap = partsMap || new Map();
this.retryOptions = {
retries: this.bucket.storage.retryOptions.maxRetries,
factor: this.bucket.storage.retryOptions.retryDelayMultiplier,
maxTimeout: this.bucket.storage.retryOptions.maxRetryDelay * 1000,
maxRetryTime: this.bucket.storage.retryOptions.totalTimeout * 1000,
};
}
/**
* Initiates a multipart upload (MPU) to the XML API and stores the resultant upload id.
*
* @returns {Promise<void>}
*/
async initiateUpload(headers = {}) {
const url = `${this.baseUrl}?uploads`;
return (0, async_retry_1.default)(async (bail) => {
try {
const res = await this.authClient.request({
headers: __classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_setGoogApiClientHeaders).call(this, headers),
method: 'POST',
url,
});
if (res.data && res.data.error) {
throw res.data.error;
}
const parsedXML = this.xmlParser.parse(res.data);
this.uploadId = parsedXML.InitiateMultipartUploadResult.UploadId;
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
}
}, this.retryOptions);
}
/**
* Uploads the provided chunk of data to the XML API using the previously created upload id.
*
* @param {number} partNumber the sequence number of this chunk.
* @param {Buffer} chunk the chunk of data to be uploaded.
* @param {string | false} validation whether or not to include the md5 hash in the headers to cause the server
* to validate the chunk was not corrupted.
* @returns {Promise<void>}
*/
async uploadPart(partNumber, chunk, validation) {
const url = `${this.baseUrl}?partNumber=${partNumber}&uploadId=${this.uploadId}`;
let headers = __classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_setGoogApiClientHeaders).call(this);
if (validation === 'md5') {
const hash = (0, crypto_1.createHash)('md5').update(chunk).digest('base64');
headers = {
'Content-MD5': hash,
};
}
return (0, async_retry_1.default)(async (bail) => {
try {
const res = await this.authClient.request({
url,
method: 'PUT',
body: chunk,
headers,
});
if (res.data && res.data.error) {
throw res.data.error;
}
this.partsMap.set(partNumber, res.headers['etag']);
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
}
}, this.retryOptions);
}
/**
* Sends the final request of the MPU to tell GCS the upload is now complete.
*
* @returns {Promise<void>}
*/
async completeUpload() {
const url = `${this.baseUrl}?uploadId=${this.uploadId}`;
const sortedMap = new Map([...this.partsMap.entries()].sort((a, b) => a[0] - b[0]));
const parts = [];
for (const entry of sortedMap.entries()) {
parts.push({ PartNumber: entry[0], ETag: entry[1] });
}
const body = `<CompleteMultipartUpload>${this.xmlBuilder.build(parts)}</CompleteMultipartUpload>`;
return (0, async_retry_1.default)(async (bail) => {
try {
const res = await this.authClient.request({
headers: __classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_setGoogApiClientHeaders).call(this),
url,
method: 'POST',
body,
});
if (res.data && res.data.error) {
throw res.data.error;
}
return res;
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
return;
}
}, this.retryOptions);
}
/**
* Aborts an multipart upload that is in progress. Once aborted, any parts in the process of being uploaded fail,
* and future requests using the upload ID fail.
*
* @returns {Promise<void>}
*/
async abortUpload() {
const url = `${this.baseUrl}?uploadId=${this.uploadId}`;
return (0, async_retry_1.default)(async (bail) => {
try {
const res = await this.authClient.request({
url,
method: 'DELETE',
});
if (res.data && res.data.error) {
throw res.data.error;
}
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
return;
}
}, this.retryOptions);
}
}
_XMLMultiPartUploadHelper_instances = new WeakSet(), _XMLMultiPartUploadHelper_setGoogApiClientHeaders = function _XMLMultiPartUploadHelper_setGoogApiClientHeaders(headers = {}) {
let headerFound = false;
let userAgentFound = false;
for (const [key, value] of Object.entries(headers)) {
if (key.toLocaleLowerCase().trim() === 'x-goog-api-client') {
headerFound = true;
// Prepend command feature to value, if not already there
if (!value.includes(GCCL_GCS_CMD_FEATURE.UPLOAD_SHARDED)) {
headers[key] =
`${value} gccl-gcs-cmd/${GCCL_GCS_CMD_FEATURE.UPLOAD_SHARDED}`;
}
}
else if (key.toLocaleLowerCase().trim() === 'user-agent') {
userAgentFound = true;
}
}
// If the header isn't present, add it
if (!headerFound) {
headers['x-goog-api-client'] = `${(0, util_js_2.getRuntimeTrackingString)()} gccl/${packageJson.version} gccl-gcs-cmd/${GCCL_GCS_CMD_FEATURE.UPLOAD_SHARDED}`;
}
// If the User-Agent isn't present, add it
if (!userAgentFound) {
headers['User-Agent'] = (0, util_js_2.getUserAgentString)();
}
return headers;
}, _XMLMultiPartUploadHelper_handleErrorResponse = function _XMLMultiPartUploadHelper_handleErrorResponse(err, bail) {
if (this.bucket.storage.retryOptions.autoRetry &&
this.bucket.storage.retryOptions.retryableErrorFn(err)) {
throw err;
}
else {
bail(err);
}
};
/**
* Create a TransferManager object to perform parallel transfer operations on a Cloud Storage bucket.
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket A {@link Bucket} instance
*
*/
class TransferManager {
constructor(bucket) {
this.bucket = bucket;
}
/**
* @typedef {object} UploadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the files.
* @property {Function} [customDestinationBuilder] A function that will take the current path of a local file
* and return a string representing a custom path to be used to upload the file to GCS.
* @property {boolean} [skipIfExists] Do not upload the file if it already exists in
* the bucket. This will set the precondition ifGenerationMatch = 0.
* @property {string} [prefix] A prefix to append to all of the uploaded files.
* @property {object} [passthroughOptions] {@link UploadOptions} Options to be passed through
* to each individual upload operation.
*
*/
/**
* Upload multiple files in parallel to the bucket. This is a convenience method
* that utilizes {@link Bucket#upload} to perform the upload.
*
* @param {array | string} [filePathsOrDirectory] An array of fully qualified paths to the files or a directory name.
* If a directory name is provided, the directory will be recursively walked and all files will be added to the upload list.
* to be uploaded to the bucket
* @param {UploadManyFilesOptions} [options] Configuration options.
* @returns {Promise<UploadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload multiple files in parallel.
* //-
* const response = await transferManager.uploadManyFiles(['/local/path/file1.txt, 'local/path/file2.txt']);
* // Your bucket now contains:
* // - "local/path/file1.txt" (with the contents of '/local/path/file1.txt')
* // - "local/path/file2.txt" (with the contents of '/local/path/file2.txt')
* const response = await transferManager.uploadManyFiles('/local/directory');
* // Your bucket will now contain all files contained in '/local/directory' maintaining the subdirectory structure.
* ```
*
*/
async uploadManyFiles(filePathsOrDirectory, options = {}) {
var _a;
if (options.skipIfExists && ((_a = options.passthroughOptions) === null || _a === void 0 ? void 0 : _a.preconditionOpts)) {
options.passthroughOptions.preconditionOpts.ifGenerationMatch = 0;
}
else if (options.skipIfExists &&
options.passthroughOptions === undefined) {
options.passthroughOptions = {
preconditionOpts: {
ifGenerationMatch: 0,
},
};
}
const limit = (0, p_limit_1.default)(options.concurrencyLimit || DEFAULT_PARALLEL_UPLOAD_LIMIT);
const promises = [];
let allPaths = [];
if (!Array.isArray(filePathsOrDirectory)) {
for await (const curPath of this.getPathsFromDirectory(filePathsOrDirectory)) {
allPaths.push(curPath);
}
}
else {
allPaths = filePathsOrDirectory;
}
for (const filePath of allPaths) {
const stat = await fs_1.promises.lstat(filePath);
if (stat.isDirectory()) {
continue;
}
const passThroughOptionsCopy = {
...options.passthroughOptions,
[util_js_1.GCCL_GCS_CMD_KEY]: GCCL_GCS_CMD_FEATURE.UPLOAD_MANY,
};
passThroughOptionsCopy.destination = options.customDestinationBuilder
? options.customDestinationBuilder(filePath, options)
: filePath.split(path.sep).join(path.posix.sep);
if (options.prefix) {
passThroughOptionsCopy.destination = path.posix.join(...options.prefix.split(path.sep), passThroughOptionsCopy.destination);
}
promises.push(limit(() => this.bucket.upload(filePath, passThroughOptionsCopy)));
}
return Promise.all(promises);
}
/**
* @typedef {object} DownloadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the files.
* @property {string} [prefix] A prefix to append to all of the downloaded files.
* @property {string} [stripPrefix] A prefix to remove from all of the downloaded files.
* @property {object} [passthroughOptions] {@link DownloadOptions} Options to be passed through
* to each individual download operation.
* @property {boolean} [skipIfExists] Do not download the file if it already exists in
* the destination.
*
*/
/**
* Download multiple files in parallel to the local filesystem. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {array | string} [filesOrFolder] An array of file name strings or file objects to be downloaded. If
* a string is provided this will be treated as a GCS prefix and all files with that prefix will be downloaded.
* @param {DownloadManyFilesOptions} [options] Configuration options. Setting options.prefix or options.stripPrefix
* or options.passthroughOptions.destination will cause the downloaded files to be written to the file system
* instead of being returned as a buffer.
* @returns {Promise<DownloadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download multiple files in parallel.
* //-
* const response = await transferManager.downloadManyFiles(['file1.txt', 'file2.txt']);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles([bucket.File('file1.txt'), bucket.File('file2.txt')]);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles('test-folder');
* // All files with GCS prefix of 'test-folder' have been downloaded.
* ```
*
*/
async downloadManyFiles(filesOrFolder, options = {}) {
const limit = (0, p_limit_1.default)(options.concurrencyLimit || DEFAULT_PARALLEL_DOWNLOAD_LIMIT);
const promises = [];
let files = [];
if (!Array.isArray(filesOrFolder)) {
const directoryFiles = await this.bucket.getFiles({
prefix: filesOrFolder,
});
files = directoryFiles[0];
}
else {
files = filesOrFolder.map(curFile => {
if (typeof curFile === 'string') {
return this.bucket.file(curFile);
}
return curFile;
});
}
const stripRegexString = options.stripPrefix
? `^${options.stripPrefix}`
: EMPTY_REGEX;
const regex = new RegExp(stripRegexString, 'g');
for (const file of files) {
const passThroughOptionsCopy = {
...options.passthroughOptions,
[util_js_1.GCCL_GCS_CMD_KEY]: GCCL_GCS_CMD_FEATURE.DOWNLOAD_MANY,
};
if (options.prefix || passThroughOptionsCopy.destination) {
passThroughOptionsCopy.destination = path.join(options.prefix || '', passThroughOptionsCopy.destination || '', file.name);
}
if (options.stripPrefix) {
passThroughOptionsCopy.destination = file.name.replace(regex, '');
}
if (options.skipIfExists &&
(0, fs_1.existsSync)(passThroughOptionsCopy.destination || '')) {
continue;
}
promises.push(limit(async () => {
const destination = passThroughOptionsCopy.destination;
if (destination && destination.endsWith(path.sep)) {
await fs_1.promises.mkdir(destination, { recursive: true });
return Promise.resolve([
Buffer.alloc(0),
]);
}
return file.download(passThroughOptionsCopy);
}));
}
return Promise.all(promises);
}
/**
* @typedef {object} DownloadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be downloaded.
* @property {string | boolean} [validation] Whether or not to perform a CRC32C validation check when download is complete.
* @property {boolean} [noReturnData] Whether or not to return the downloaded data. A `true` value here would be useful for files with a size that will not fit into memory.
*
*/
/**
* Download a large file in chunks utilizing parallel download operations. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {File | string} fileOrName {@link File} to download.
* @param {DownloadFileInChunksOptions} [options] Configuration options.
* @returns {Promise<void | DownloadResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.downloadFileInChunks(bucket.file('large-file.txt');
* // Your local directory now contains:
* // - "large-file.txt" (with the contents from my-bucket.large-file.txt)
* ```
*
*/
async downloadFileInChunks(fileOrName, options = {}) {
let chunkSize = options.chunkSizeBytes || DOWNLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE;
let limit = (0, p_limit_1.default)(options.concurrencyLimit || DEFAULT_PARALLEL_CHUNKED_DOWNLOAD_LIMIT);
const noReturnData = Boolean(options.noReturnData);
const promises = [];
const file = typeof fileOrName === 'string'
? this.bucket.file(fileOrName)
: fileOrName;
const fileInfo = await file.get();
const size = parseInt(fileInfo[0].metadata.size.toString());
// If the file size does not meet the threshold download it as a single chunk.
if (size < DOWNLOAD_IN_CHUNKS_FILE_SIZE_THRESHOLD) {
limit = (0, p_limit_1.default)(1);
chunkSize = size;
}
let start = 0;
const filePath = options.destination || path.basename(file.name);
const fileToWrite = await fs_1.promises.open(filePath, 'w');
while (start < size) {
const chunkStart = start;
let chunkEnd = start + chunkSize - 1;
chunkEnd = chunkEnd > size ? size : chunkEnd;
promises.push(limit(async () => {
const resp = await file.download({
start: chunkStart,
end: chunkEnd,
[util_js_1.GCCL_GCS_CMD_KEY]: GCCL_GCS_CMD_FEATURE.DOWNLOAD_SHARDED,
});
const result = await fileToWrite.write(resp[0], 0, resp[0].length, chunkStart);
if (noReturnData)
return;
return result.buffer;
}));
start += chunkSize;
}
let chunks;
try {
chunks = await Promise.all(promises);
}
finally {
await fileToWrite.close();
}
if (options.validation === 'crc32c' && fileInfo[0].metadata.crc32c) {
const downloadedCrc32C = await crc32c_js_1.CRC32C.fromFile(filePath);
if (!downloadedCrc32C.validate(fileInfo[0].metadata.crc32c)) {
const mismatchError = new file_js_1.RequestError(file_js_1.FileExceptionMessages.DOWNLOAD_MISMATCH);
mismatchError.code = 'CONTENT_DOWNLOAD_MISMATCH';
throw mismatchError;
}
}
if (noReturnData)
return;
return [Buffer.concat(chunks, size)];
}
/**
* @typedef {object} UploadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be uploaded.
* @property {string} [uploadName] Name of the file when saving to GCS. If omitted the name is taken from the file path.
* @property {number} [maxQueueSize] The number of chunks to be uploaded to hold in memory concurrently. If not specified
* defaults to the specified concurrency limit.
* @property {string} [uploadId] If specified attempts to resume a previous upload.
* @property {Map} [partsMap] If specified alongside uploadId, attempts to resume a previous upload from the last chunk
* specified in partsMap
* @property {object} [headers] headers to be sent when initiating the multipart upload.
* See {@link https://cloud.google.com/storage/docs/xml-api/post-object-multipart#request_headers| Request Headers: Initiate a Multipart Upload}
* @property {boolean} [autoAbortFailure] boolean to indicate if an in progress upload session will be automatically aborted upon failure. If not set,
* failures will be automatically aborted.
*
*/
/**
* Upload a large file in chunks utilizing parallel upload operations. If the upload fails, an uploadId and
* map containing all the successfully uploaded parts will be returned to the caller. These arguments can be used to
* resume the upload.
*
* @param {string} [filePath] The path of the file to be uploaded
* @param {UploadFileInChunksOptions} [options] Configuration options.
* @param {MultiPartHelperGenerator} [generator] A function that will return a type that implements the MPU interface. Most users will not need to use this.
* @returns {Promise<void>} If successful a promise resolving to void, otherwise a error containing the message, uploadId, and parts map.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.uploadFileInChunks('large-file.txt');
* // Your bucket now contains:
* // - "large-file.txt"
* ```
*
*
*/
async uploadFileInChunks(filePath, options = {}, generator = defaultMultiPartGenerator) {
const chunkSize = options.chunkSizeBytes || UPLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE;
const limit = (0, p_limit_1.default)(options.concurrencyLimit || DEFAULT_PARALLEL_CHUNKED_UPLOAD_LIMIT);
const maxQueueSize = options.maxQueueSize ||
options.concurrencyLimit ||
DEFAULT_PARALLEL_CHUNKED_UPLOAD_LIMIT;
const fileName = options.uploadName || path.basename(filePath);
const mpuHelper = generator(this.bucket, fileName, options.uploadId, options.partsMap);
let partNumber = 1;
let promises = [];
try {
if (options.uploadId === undefined) {
await mpuHelper.initiateUpload(options.headers);
}
const startOrResumptionByte = mpuHelper.partsMap.size * chunkSize;
const readStream = (0, fs_1.createReadStream)(filePath, {
highWaterMark: chunkSize,
start: startOrResumptionByte,
});
// p-limit only limits the number of running promises. We do not want to hold an entire
// large file in memory at once so promises acts a queue that will hold only maxQueueSize in memory.
for await (const curChunk of readStream) {
if (promises.length >= maxQueueSize) {
await Promise.all(promises);
promises = [];
}
promises.push(limit(() => mpuHelper.uploadPart(partNumber++, curChunk, options.validation)));
}
await Promise.all(promises);
return await mpuHelper.completeUpload();
}
catch (e) {
if ((options.autoAbortFailure === undefined || options.autoAbortFailure) &&
mpuHelper.uploadId) {
try {
await mpuHelper.abortUpload();
return;
}
catch (e) {
throw new MultiPartUploadError(e.message, mpuHelper.uploadId, mpuHelper.partsMap);
}
}
throw new MultiPartUploadError(e.message, mpuHelper.uploadId, mpuHelper.partsMap);
}
}
async *getPathsFromDirectory(directory) {
const filesAndSubdirectories = await fs_1.promises.readdir(directory, {
withFileTypes: true,
});
for (const curFileOrDirectory of filesAndSubdirectories) {
const fullPath = path.join(directory, curFileOrDirectory.name);
curFileOrDirectory.isDirectory()
? yield* this.getPathsFromDirectory(fullPath)
: yield fullPath;
}
}
}
exports.TransferManager = TransferManager;

View File

@@ -0,0 +1,85 @@
import * as querystring from 'querystring';
import { PassThrough } from 'stream';
export declare function normalize<T = {}, U = Function>(optionsOrCallback?: T | U, cb?: U): {
options: T;
callback: U;
};
/**
* Flatten an object into an Array of arrays, [[key, value], ..].
* Implements Object.entries() for Node.js <8
* @internal
*/
export declare function objectEntries<T>(obj: {
[key: string]: T;
}): Array<[string, T]>;
/**
* Encode `str` with encodeURIComponent, plus these
* reserved characters: `! * ' ( )`.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent| MDN: fixedEncodeURIComponent}
*
* @param {string} str The URI component to encode.
* @return {string} The encoded string.
*/
export declare function fixedEncodeURIComponent(str: string): string;
/**
* URI encode `uri` for generating signed URLs, using fixedEncodeURIComponent.
*
* Encode every byte except `A-Z a-Z 0-9 ~ - . _`.
*
* @param {string} uri The URI to encode.
* @param [boolean=false] encodeSlash If `true`, the "/" character is not encoded.
* @return {string} The encoded string.
*/
export declare function encodeURI(uri: string, encodeSlash: boolean): string;
/**
* Serialize an object to a URL query string using util.encodeURI(uri, true).
* @param {string} url The object to serialize.
* @return {string} Serialized string.
*/
export declare function qsStringify(qs: querystring.ParsedUrlQueryInput): string;
export declare function objectKeyToLowercase<T>(object: {
[key: string]: T;
}): {
[key: string]: T;
};
/**
* JSON encode str, with unicode \u+ representation.
* @param {object} obj The object to encode.
* @return {string} Serialized string.
*/
export declare function unicodeJSONStringify(obj: object): string;
/**
* Converts the given objects keys to snake_case
* @param {object} obj object to convert keys to snake case.
* @returns {object} object with keys converted to snake case.
*/
export declare function convertObjKeysToSnakeCase(obj: object): object;
/**
* Formats the provided date object as a UTC ISO string.
* @param {Date} dateTimeToFormat date object to be formatted.
* @param {boolean} includeTime flag to include hours, minutes, seconds in output.
* @param {string} dateDelimiter delimiter between date components.
* @param {string} timeDelimiter delimiter between time components.
* @returns {string} UTC ISO format of provided date object.
*/
export declare function formatAsUTCISO(dateTimeToFormat: Date, includeTime?: boolean, dateDelimiter?: string, timeDelimiter?: string): string;
/**
* Examines the runtime environment and returns the appropriate tracking string.
* @returns {string} metrics tracking string based on the current runtime environment.
*/
export declare function getRuntimeTrackingString(): string;
/**
* Looks at package.json and creates the user-agent string to be applied to request headers.
* @returns {string} user agent string.
*/
export declare function getUserAgentString(): string;
export declare function getDirName(): string;
export declare function getModuleFormat(): "ESM" | "CJS";
export declare class PassThroughShim extends PassThrough {
private shouldEmitReading;
private shouldEmitWriting;
_read(size: number): void;
_write(chunk: never, encoding: BufferEncoding, callback: (error?: Error | null | undefined) => void): void;
_final(callback: (error?: Error | null | undefined) => void): void;
}

View File

@@ -0,0 +1,280 @@
"use strict";
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __createBinding = this && this.__createBinding || (Object.create ? function (o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = {
enumerable: true,
get: function () {
return m[k];
}
};
}
Object.defineProperty(o, k2, desc);
} : function (o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
});
var __setModuleDefault = this && this.__setModuleDefault || (Object.create ? function (o, v) {
Object.defineProperty(o, "default", {
enumerable: true,
value: v
});
} : function (o, v) {
o["default"] = v;
});
var __importStar = this && this.__importStar || function () {
var ownKeys = function (o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
}();
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.PassThroughShim = void 0;
exports.normalize = normalize;
exports.objectEntries = objectEntries;
exports.fixedEncodeURIComponent = fixedEncodeURIComponent;
exports.encodeURI = encodeURI;
exports.qsStringify = qsStringify;
exports.objectKeyToLowercase = objectKeyToLowercase;
exports.unicodeJSONStringify = unicodeJSONStringify;
exports.convertObjKeysToSnakeCase = convertObjKeysToSnakeCase;
exports.formatAsUTCISO = formatAsUTCISO;
exports.getRuntimeTrackingString = getRuntimeTrackingString;
exports.getUserAgentString = getUserAgentString;
exports.getDirName = getDirName;
exports.getModuleFormat = getModuleFormat;
const path = __importStar(require("path"));
const querystring = __importStar(require("querystring"));
const stream_1 = require("stream");
const url = __importStar(require("url"));
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
const package_json_helper_cjs_1 = require("./package-json-helper.cjs");
// Done to avoid a problem with mangling of identifiers when using esModuleInterop
const fileURLToPath = url.fileURLToPath;
const isEsm = false;
function normalize(optionsOrCallback, cb) {
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : {};
const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : cb;
return {
options,
callback
};
}
/**
* Flatten an object into an Array of arrays, [[key, value], ..].
* Implements Object.entries() for Node.js <8
* @internal
*/
function objectEntries(obj) {
return Object.keys(obj).map(key => [key, obj[key]]);
}
/**
* Encode `str` with encodeURIComponent, plus these
* reserved characters: `! * ' ( )`.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent| MDN: fixedEncodeURIComponent}
*
* @param {string} str The URI component to encode.
* @return {string} The encoded string.
*/
function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, c => '%' + c.charCodeAt(0).toString(16).toUpperCase());
}
/**
* URI encode `uri` for generating signed URLs, using fixedEncodeURIComponent.
*
* Encode every byte except `A-Z a-Z 0-9 ~ - . _`.
*
* @param {string} uri The URI to encode.
* @param [boolean=false] encodeSlash If `true`, the "/" character is not encoded.
* @return {string} The encoded string.
*/
function encodeURI(uri, encodeSlash) {
// Split the string by `/`, and conditionally rejoin them with either
// %2F if encodeSlash is `true`, or '/' if `false`.
return uri.split('/').map(fixedEncodeURIComponent).join(encodeSlash ? '%2F' : '/');
}
/**
* Serialize an object to a URL query string using util.encodeURI(uri, true).
* @param {string} url The object to serialize.
* @return {string} Serialized string.
*/
function qsStringify(qs) {
return querystring.stringify(qs, '&', '=', {
encodeURIComponent: component => encodeURI(component, true)
});
}
function objectKeyToLowercase(object) {
const newObj = {};
for (let key of Object.keys(object)) {
const value = object[key];
key = key.toLowerCase();
newObj[key] = value;
}
return newObj;
}
/**
* JSON encode str, with unicode \u+ representation.
* @param {object} obj The object to encode.
* @return {string} Serialized string.
*/
function unicodeJSONStringify(obj) {
return JSON.stringify(obj).replace(/[\u0080-\uFFFF]/g, char => '\\u' + ('0000' + char.charCodeAt(0).toString(16)).slice(-4));
}
/**
* Converts the given objects keys to snake_case
* @param {object} obj object to convert keys to snake case.
* @returns {object} object with keys converted to snake case.
*/
function convertObjKeysToSnakeCase(obj) {
if (obj instanceof Date || obj instanceof RegExp) {
return obj;
}
if (Array.isArray(obj)) {
return obj.map(convertObjKeysToSnakeCase);
}
if (obj instanceof Object) {
return Object.keys(obj).reduce((acc, cur) => {
const s = cur[0].toLocaleLowerCase() + cur.slice(1).replace(/([A-Z]+)/g, (match, p1) => {
return `_${p1.toLowerCase()}`;
});
acc[s] = convertObjKeysToSnakeCase(obj[cur]);
return acc;
}, Object());
}
return obj;
}
/**
* Formats the provided date object as a UTC ISO string.
* @param {Date} dateTimeToFormat date object to be formatted.
* @param {boolean} includeTime flag to include hours, minutes, seconds in output.
* @param {string} dateDelimiter delimiter between date components.
* @param {string} timeDelimiter delimiter between time components.
* @returns {string} UTC ISO format of provided date object.
*/
function formatAsUTCISO(dateTimeToFormat, includeTime = false, dateDelimiter = '', timeDelimiter = '') {
const year = dateTimeToFormat.getUTCFullYear();
const month = dateTimeToFormat.getUTCMonth() + 1;
const day = dateTimeToFormat.getUTCDate();
const hour = dateTimeToFormat.getUTCHours();
const minute = dateTimeToFormat.getUTCMinutes();
const second = dateTimeToFormat.getUTCSeconds();
let resultString = `${year.toString().padStart(4, '0')}${dateDelimiter}${month.toString().padStart(2, '0')}${dateDelimiter}${day.toString().padStart(2, '0')}`;
if (includeTime) {
resultString = `${resultString}T${hour.toString().padStart(2, '0')}${timeDelimiter}${minute.toString().padStart(2, '0')}${timeDelimiter}${second.toString().padStart(2, '0')}Z`;
}
return resultString;
}
/**
* Examines the runtime environment and returns the appropriate tracking string.
* @returns {string} metrics tracking string based on the current runtime environment.
*/
function getRuntimeTrackingString() {
if (
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
globalThis.Deno &&
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
globalThis.Deno.version &&
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
globalThis.Deno.version.deno) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
return `gl-deno/${globalThis.Deno.version.deno}`;
} else {
return `gl-node/${process.versions.node}`;
}
}
/**
* Looks at package.json and creates the user-agent string to be applied to request headers.
* @returns {string} user agent string.
*/
function getUserAgentString() {
const pkg = (0, package_json_helper_cjs_1.getPackageJSON)();
const hyphenatedPackageName = pkg.name.replace('@google-cloud', 'gcloud-node') // For legacy purposes.
.replace('/', '-'); // For UA spec-compliance purposes.
return hyphenatedPackageName + '/' + pkg.version;
}
function getDirName() {
let dirToUse = '';
try {
dirToUse = __dirname;
} catch (e) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
dirToUse = __dirname;
}
return dirToUse;
}
function getModuleFormat() {
return isEsm ? 'ESM' : 'CJS';
}
class PassThroughShim extends stream_1.PassThrough {
constructor() {
super(...arguments);
this.shouldEmitReading = true;
this.shouldEmitWriting = true;
}
_read(size) {
if (this.shouldEmitReading) {
this.emit('reading');
this.shouldEmitReading = false;
}
super._read(size);
}
_write(chunk, encoding, callback) {
if (this.shouldEmitWriting) {
this.emit('writing');
this.shouldEmitWriting = false;
}
// Per the nodejs documentation, callback must be invoked on the next tick
process.nextTick(() => {
super._write(chunk, encoding, callback);
});
}
_final(callback) {
// If the stream is empty (i.e. empty file) final will be invoked before _read / _write
// and we should still emit the proper events.
if (this.shouldEmitReading) {
this.emit('reading');
this.shouldEmitReading = false;
}
if (this.shouldEmitWriting) {
this.emit('writing');
this.shouldEmitWriting = false;
}
callback(null);
}
}
exports.PassThroughShim = PassThroughShim;

View File

@@ -0,0 +1,165 @@
import { BodyResponseCallback, DecorateRequestOptions, BaseMetadata } from './nodejs-common/index.js';
export interface AclOptions {
pathPrefix: string;
request: (reqOpts: DecorateRequestOptions, callback: BodyResponseCallback) => void;
}
export type GetAclResponse = [
AccessControlObject | AccessControlObject[],
AclMetadata
];
export interface GetAclCallback {
(err: Error | null, acl?: AccessControlObject | AccessControlObject[] | null, apiResponse?: AclMetadata): void;
}
export interface GetAclOptions {
entity: string;
generation?: number;
userProject?: string;
}
export interface UpdateAclOptions {
entity: string;
role: string;
generation?: number;
userProject?: string;
}
export type UpdateAclResponse = [AccessControlObject, AclMetadata];
export interface UpdateAclCallback {
(err: Error | null, acl?: AccessControlObject | null, apiResponse?: AclMetadata): void;
}
export interface AddAclOptions {
entity: string;
role: string;
generation?: number;
userProject?: string;
}
export type AddAclResponse = [AccessControlObject, AclMetadata];
export interface AddAclCallback {
(err: Error | null, acl?: AccessControlObject | null, apiResponse?: AclMetadata): void;
}
export type RemoveAclResponse = [AclMetadata];
export interface RemoveAclCallback {
(err: Error | null, apiResponse?: AclMetadata): void;
}
export interface RemoveAclOptions {
entity: string;
generation?: number;
userProject?: string;
}
export interface AccessControlObject {
entity: string;
role: string;
projectTeam: string;
}
export interface AclMetadata extends BaseMetadata {
bucket?: string;
domain?: string;
entity?: string;
entityId?: string;
generation?: string;
object?: string;
projectTeam?: {
projectNumber?: string;
team?: 'editors' | 'owners' | 'viewers';
};
role?: 'OWNER' | 'READER' | 'WRITER' | 'FULL_CONTROL';
[key: string]: unknown;
}
/**
* Attach functionality to a {@link Storage.acl} instance. This will add an
* object for each role group (owners, readers, and writers), with each object
* containing methods to add or delete a type of entity.
*
* As an example, here are a few methods that are created.
*
* myBucket.acl.readers.deleteGroup('groupId', function(err) {});
*
* myBucket.acl.owners.addUser('email@example.com', function(err, acl) {});
*
* myBucket.acl.writers.addDomain('example.com', function(err, acl) {});
*
* @private
*/
declare class AclRoleAccessorMethods {
private static accessMethods;
private static entities;
private static roles;
owners: {};
readers: {};
writers: {};
constructor();
_assignAccessMethods(role: string): void;
}
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against an
* object or bucket (for example, `READ` or `WRITE`); the entity defines who the
* permission applies to (for example, a specific user or group of users).
*
* Where an `entity` value is accepted, we follow the format the Cloud Storage
* API expects.
*
* Refer to
* https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls
* for the most up-to-date values.
*
* - `user-userId`
* - `user-email`
* - `group-groupId`
* - `group-email`
* - `domain-domain`
* - `project-team-projectId`
* - `allUsers`
* - `allAuthenticatedUsers`
*
* Examples:
*
* - The user "liz@example.com" would be `user-liz@example.com`.
* - The group "example@googlegroups.com" would be
* `group-example@googlegroups.com`.
* - To refer to all members of the Google Apps for Business domain
* "example.com", the entity would be `domain-example.com`.
*
* For more detailed information, see
* {@link http://goo.gl/6qBBPO| About Access Control Lists}.
*
* @constructor Acl
* @mixin
* @param {object} options Configuration options.
*/
declare class Acl extends AclRoleAccessorMethods {
default: Acl;
pathPrefix: string;
request_: (reqOpts: DecorateRequestOptions, callback: BodyResponseCallback) => void;
constructor(options: AclOptions);
add(options: AddAclOptions): Promise<AddAclResponse>;
add(options: AddAclOptions, callback: AddAclCallback): void;
delete(options: RemoveAclOptions): Promise<RemoveAclResponse>;
delete(options: RemoveAclOptions, callback: RemoveAclCallback): void;
get(options?: GetAclOptions): Promise<GetAclResponse>;
get(options: GetAclOptions, callback: GetAclCallback): void;
get(callback: GetAclCallback): void;
update(options: UpdateAclOptions): Promise<UpdateAclResponse>;
update(options: UpdateAclOptions, callback: UpdateAclCallback): void;
/**
* Transform API responses to a consistent object format.
*
* @private
*/
makeAclObject_(accessControlObject: AccessControlObject): AccessControlObject;
/**
* Patch requests up to the bucket's request object.
*
* @private
*
* @param {string} method Action.
* @param {string} path Request path.
* @param {*} query Request query object.
* @param {*} body Request body contents.
* @param {function} callback Callback function.
*/
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
}
export { Acl, AclRoleAccessorMethods };

View File

@@ -0,0 +1,714 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { promisifyAll } from '@google-cloud/promisify';
/**
* Attach functionality to a {@link Storage.acl} instance. This will add an
* object for each role group (owners, readers, and writers), with each object
* containing methods to add or delete a type of entity.
*
* As an example, here are a few methods that are created.
*
* myBucket.acl.readers.deleteGroup('groupId', function(err) {});
*
* myBucket.acl.owners.addUser('email@example.com', function(err, acl) {});
*
* myBucket.acl.writers.addDomain('example.com', function(err, acl) {});
*
* @private
*/
class AclRoleAccessorMethods {
constructor() {
this.owners = {};
this.readers = {};
this.writers = {};
/**
* An object of convenience methods to add or delete owner ACL permissions
* for a given entity.
*
* The supported methods include:
*
* - `myFile.acl.owners.addAllAuthenticatedUsers`
* - `myFile.acl.owners.deleteAllAuthenticatedUsers`
* - `myFile.acl.owners.addAllUsers`
* - `myFile.acl.owners.deleteAllUsers`
* - `myFile.acl.owners.addDomain`
* - `myFile.acl.owners.deleteDomain`
* - `myFile.acl.owners.addGroup`
* - `myFile.acl.owners.deleteGroup`
* - `myFile.acl.owners.addProject`
* - `myFile.acl.owners.deleteProject`
* - `myFile.acl.owners.addUser`
* - `myFile.acl.owners.deleteUser`
*
* @name Acl#owners
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* //-
* // Add a user as an owner of a file.
* //-
* const myBucket = gcs.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
* myFile.acl.owners.addUser('email@example.com', function(err, aclObject)
* {});
*
* //-
* // For reference, the above command is the same as running the following.
* //-
* myFile.acl.add({
* entity: 'user-email@example.com',
* role: gcs.acl.OWNER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.owners.addUser('email@example.com').then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
this.owners = {};
/**
* An object of convenience methods to add or delete reader ACL permissions
* for a given entity.
*
* The supported methods include:
*
* - `myFile.acl.readers.addAllAuthenticatedUsers`
* - `myFile.acl.readers.deleteAllAuthenticatedUsers`
* - `myFile.acl.readers.addAllUsers`
* - `myFile.acl.readers.deleteAllUsers`
* - `myFile.acl.readers.addDomain`
* - `myFile.acl.readers.deleteDomain`
* - `myFile.acl.readers.addGroup`
* - `myFile.acl.readers.deleteGroup`
* - `myFile.acl.readers.addProject`
* - `myFile.acl.readers.deleteProject`
* - `myFile.acl.readers.addUser`
* - `myFile.acl.readers.deleteUser`
*
* @name Acl#readers
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* //-
* // Add a user as a reader of a file.
* //-
* myFile.acl.readers.addUser('email@example.com', function(err, aclObject)
* {});
*
* //-
* // For reference, the above command is the same as running the following.
* //-
* myFile.acl.add({
* entity: 'user-email@example.com',
* role: gcs.acl.READER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.readers.addUser('email@example.com').then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
this.readers = {};
/**
* An object of convenience methods to add or delete writer ACL permissions
* for a given entity.
*
* The supported methods include:
*
* - `myFile.acl.writers.addAllAuthenticatedUsers`
* - `myFile.acl.writers.deleteAllAuthenticatedUsers`
* - `myFile.acl.writers.addAllUsers`
* - `myFile.acl.writers.deleteAllUsers`
* - `myFile.acl.writers.addDomain`
* - `myFile.acl.writers.deleteDomain`
* - `myFile.acl.writers.addGroup`
* - `myFile.acl.writers.deleteGroup`
* - `myFile.acl.writers.addProject`
* - `myFile.acl.writers.deleteProject`
* - `myFile.acl.writers.addUser`
* - `myFile.acl.writers.deleteUser`
*
* @name Acl#writers
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* //-
* // Add a user as a writer of a file.
* //-
* myFile.acl.writers.addUser('email@example.com', function(err, aclObject)
* {});
*
* //-
* // For reference, the above command is the same as running the following.
* //-
* myFile.acl.add({
* entity: 'user-email@example.com',
* role: gcs.acl.WRITER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.writers.addUser('email@example.com').then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
this.writers = {};
AclRoleAccessorMethods.roles.forEach(this._assignAccessMethods.bind(this));
}
_assignAccessMethods(role) {
const accessMethods = AclRoleAccessorMethods.accessMethods;
const entities = AclRoleAccessorMethods.entities;
const roleGroup = role.toLowerCase() + 's';
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this[roleGroup] = entities.reduce((acc, entity) => {
const isPrefix = entity.charAt(entity.length - 1) === '-';
accessMethods.forEach(accessMethod => {
let method = accessMethod + entity[0].toUpperCase() + entity.substring(1);
if (isPrefix) {
method = method.replace('-', '');
}
// Wrap the parent accessor method (e.g. `add` or `delete`) to avoid the
// more complex API of specifying an `entity` and `role`.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
acc[method] = (entityId, options, callback) => {
let apiEntity;
if (typeof options === 'function') {
callback = options;
options = {};
}
if (isPrefix) {
apiEntity = entity + entityId;
}
else {
// If the entity is not a prefix, it is a special entity group
// that does not require further details. The accessor methods
// only accept a callback.
apiEntity = entity;
callback = entityId;
}
options = Object.assign({
entity: apiEntity,
role,
}, options);
const args = [options];
if (typeof callback === 'function') {
args.push(callback);
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return this[accessMethod].apply(this, args);
};
});
return acc;
}, {});
}
}
AclRoleAccessorMethods.accessMethods = ['add', 'delete'];
AclRoleAccessorMethods.entities = [
// Special entity groups that do not require further specification.
'allAuthenticatedUsers',
'allUsers',
// Entity groups that require specification, e.g. `user-email@example.com`.
'domain-',
'group-',
'project-',
'user-',
];
AclRoleAccessorMethods.roles = ['OWNER', 'READER', 'WRITER'];
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against an
* object or bucket (for example, `READ` or `WRITE`); the entity defines who the
* permission applies to (for example, a specific user or group of users).
*
* Where an `entity` value is accepted, we follow the format the Cloud Storage
* API expects.
*
* Refer to
* https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls
* for the most up-to-date values.
*
* - `user-userId`
* - `user-email`
* - `group-groupId`
* - `group-email`
* - `domain-domain`
* - `project-team-projectId`
* - `allUsers`
* - `allAuthenticatedUsers`
*
* Examples:
*
* - The user "liz@example.com" would be `user-liz@example.com`.
* - The group "example@googlegroups.com" would be
* `group-example@googlegroups.com`.
* - To refer to all members of the Google Apps for Business domain
* "example.com", the entity would be `domain-example.com`.
*
* For more detailed information, see
* {@link http://goo.gl/6qBBPO| About Access Control Lists}.
*
* @constructor Acl
* @mixin
* @param {object} options Configuration options.
*/
class Acl extends AclRoleAccessorMethods {
constructor(options) {
super();
this.pathPrefix = options.pathPrefix;
this.request_ = options.request;
}
/**
* @typedef {array} AddAclResponse
* @property {object} 0 The Acl Objects.
* @property {object} 1 The full API response.
*/
/**
* @callback AddAclCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The Acl Objects.
* @param {object} apiResponse The full API response.
*/
/**
* Add access controls on a {@link Bucket} or {@link File}.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/insert| BucketAccessControls: insert API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/insert| ObjectAccessControls: insert API Documentation}
*
* @param {object} options Configuration options.
* @param {string} options.entity Whose permissions will be added.
* @param {string} options.role Permissions allowed for the defined entity.
* See {@link https://cloud.google.com/storage/docs/access-control Access
* Control}.
* @param {number} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {AddAclCallback} [callback] Callback function.
* @returns {Promise<AddAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* const options = {
* entity: 'user-useremail@example.com',
* role: gcs.acl.OWNER_ROLE
* };
*
* myBucket.acl.add(options, function(err, aclObject, apiResponse) {});
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* // Here is how you would grant ownership permissions to a user on a
* specific
* // revision of a file.
* //-
* myFile.acl.add({
* entity: 'user-useremail@example.com',
* role: gcs.acl.OWNER_ROLE,
* generation: 1
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myBucket.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_file_owner
* Example of adding an owner to a file:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_owner
* Example of adding an owner to a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_default_owner
* Example of adding a default owner to a bucket:
*/
add(options, callback) {
const query = {};
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
this.request({
method: 'POST',
uri: '',
qs: query,
maxRetries: 0, //explicitly set this value since this is a non-idempotent function
json: {
entity: options.entity,
role: options.role.toUpperCase(),
},
}, (err, resp) => {
if (err) {
callback(err, null, resp);
return;
}
callback(null, this.makeAclObject_(resp), resp);
});
}
/**
* @typedef {array} RemoveAclResponse
* @property {object} 0 The full API response.
*/
/**
* @callback RemoveAclCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
/**
* Delete access controls on a {@link Bucket} or {@link File}.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/delete| BucketAccessControls: delete API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/delete| ObjectAccessControls: delete API Documentation}
*
* @param {object} options Configuration object.
* @param {string} options.entity Whose permissions will be revoked.
* @param {int} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {RemoveAclCallback} callback The callback function.
* @returns {Promise<RemoveAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* myBucket.acl.delete({
* entity: 'user-useremail@example.com'
* }, function(err, apiResponse) {});
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* //-
* myFile.acl.delete({
* entity: 'user-useremail@example.com',
* generation: 1
* }, function(err, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.delete().then(function(data) {
* const apiResponse = data[0];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_owner
* Example of removing an owner from a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_default_owner
* Example of removing a default owner from a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_file_owner
* Example of removing an owner from a bucket:
*/
delete(options, callback) {
const query = {};
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
this.request({
method: 'DELETE',
uri: '/' + encodeURIComponent(options.entity),
qs: query,
}, (err, resp) => {
callback(err, resp);
});
}
/**
* @typedef {array} GetAclResponse
* @property {object|object[]} 0 Single or array of Acl Objects.
* @property {object} 1 The full API response.
*/
/**
* @callback GetAclCallback
* @param {?Error} err Request error, if any.
* @param {object|object[]} acl Single or array of Acl Objects.
* @param {object} apiResponse The full API response.
*/
/**
* Get access controls on a {@link Bucket} or {@link File}. If
* an entity is omitted, you will receive an array of all applicable access
* controls.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/get| BucketAccessControls: get API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/get| ObjectAccessControls: get API Documentation}
*
* @param {object|function} [options] Configuration options. If you want to
* receive a list of all access controls, pass the callback function as
* the only argument.
* @param {string} options.entity Whose permissions will be fetched.
* @param {number} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {GetAclCallback} [callback] Callback function.
* @returns {Promise<GetAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* myBucket.acl.get({
* entity: 'user-useremail@example.com'
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // Get all access controls.
* //-
* myBucket.acl.get(function(err, aclObjects, apiResponse) {
* // aclObjects = [
* // {
* // entity: 'user-useremail@example.com',
* // role: 'owner'
* // }
* // ]
* });
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* //-
* myFile.acl.get({
* entity: 'user-useremail@example.com',
* generation: 1
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myBucket.acl.get().then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_file_acl
* Example of printing a file's ACL:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_file_acl_for_user
* Example of printing a file's ACL for a specific user:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl
* Example of printing a bucket's ACL:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl_for_user
* Example of printing a bucket's ACL for a specific user:
*/
get(optionsOrCallback, cb) {
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : null;
const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : cb;
let path = '';
const query = {};
if (options) {
path = '/' + encodeURIComponent(options.entity);
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
}
this.request({
uri: path,
qs: query,
}, (err, resp) => {
if (err) {
callback(err, null, resp);
return;
}
let results;
if (resp.items) {
results = resp.items.map(this.makeAclObject_);
}
else {
results = this.makeAclObject_(resp);
}
callback(null, results, resp);
});
}
/**
* @typedef {array} UpdateAclResponse
* @property {object} 0 The updated Acl Objects.
* @property {object} 1 The full API response.
*/
/**
* @callback UpdateAclCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The updated Acl Objects.
* @param {object} apiResponse The full API response.
*/
/**
* Update access controls on a {@link Bucket} or {@link File}.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/update| BucketAccessControls: update API Documentation}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/update| ObjectAccessControls: update API Documentation}
*
* @param {object} options Configuration options.
* @param {string} options.entity Whose permissions will be updated.
* @param {string} options.role Permissions allowed for the defined entity.
* See {@link Storage.acl}.
* @param {number} [options.generation] **File Objects Only** Select a specific
* revision of this file (as opposed to the latest version, the default).
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {UpdateAclCallback} [callback] Callback function.
* @returns {Promise<UpdateAclResponse>}
*
* @example
* ```
* const storage = require('@google-cloud/storage')();
* const myBucket = storage.bucket('my-bucket');
* const myFile = myBucket.file('my-file');
*
* const options = {
* entity: 'user-useremail@example.com',
* role: gcs.acl.WRITER_ROLE
* };
*
* myBucket.acl.update(options, function(err, aclObject, apiResponse) {});
*
* //-
* // For file ACL operations, you can also specify a `generation` property.
* //-
* myFile.acl.update({
* entity: 'user-useremail@example.com',
* role: gcs.acl.WRITER_ROLE,
* generation: 1
* }, function(err, aclObject, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myFile.acl.update(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
update(options, callback) {
const query = {};
if (options.generation) {
query.generation = options.generation;
}
if (options.userProject) {
query.userProject = options.userProject;
}
this.request({
method: 'PUT',
uri: '/' + encodeURIComponent(options.entity),
qs: query,
json: {
role: options.role.toUpperCase(),
},
}, (err, resp) => {
if (err) {
callback(err, null, resp);
return;
}
callback(null, this.makeAclObject_(resp), resp);
});
}
/**
* Transform API responses to a consistent object format.
*
* @private
*/
makeAclObject_(accessControlObject) {
const obj = {
entity: accessControlObject.entity,
role: accessControlObject.role,
};
if (accessControlObject.projectTeam) {
obj.projectTeam = accessControlObject.projectTeam;
}
return obj;
}
/**
* Patch requests up to the bucket's request object.
*
* @private
*
* @param {string} method Action.
* @param {string} path Request path.
* @param {*} query Request query object.
* @param {*} body Request body contents.
* @param {function} callback Callback function.
*/
request(reqOpts, callback) {
reqOpts.uri = this.pathPrefix + reqOpts.uri;
this.request_(reqOpts, callback);
}
}
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
promisifyAll(Acl, {
exclude: ['request'],
});
export { Acl, AclRoleAccessorMethods };

View File

@@ -0,0 +1,824 @@
import { ApiError, BodyResponseCallback, DecorateRequestOptions, DeleteCallback, ExistsCallback, GetConfig, MetadataCallback, ServiceObject, SetMetadataResponse } from './nodejs-common/index.js';
import { RequestResponse } from './nodejs-common/service-object.js';
import * as http from 'http';
import { Acl, AclMetadata } from './acl.js';
import { Channel } from './channel.js';
import { File, FileOptions, CreateResumableUploadOptions, CreateWriteStreamOptions } from './file.js';
import { Iam } from './iam.js';
import { Notification } from './notification.js';
import { Storage, Cors, PreconditionOptions, BucketOptions } from './storage.js';
import { GetSignedUrlResponse, GetSignedUrlCallback, SignerGetSignedUrlConfig, URLSigner, Query } from './signer.js';
import { Readable } from 'stream';
import { CRC32CValidatorGenerator } from './crc32c.js';
import { URL } from 'url';
import { BaseMetadata, SetMetadataOptions } from './nodejs-common/service-object.js';
export type GetFilesResponse = [
File[],
(GetFilesOptions | {}) & Partial<Pick<GetFilesOptions, 'pageToken'>>,
unknown
];
export interface GetFilesCallback {
(err: Error | null, files?: File[], nextQuery?: {}, apiResponse?: unknown): void;
}
interface WatchAllOptions {
delimiter?: string;
maxResults?: number;
pageToken?: string;
prefix?: string;
projection?: string;
userProject?: string;
versions?: boolean;
}
export interface AddLifecycleRuleOptions extends PreconditionOptions {
append?: boolean;
}
export interface LifecycleAction {
type: 'Delete' | 'SetStorageClass' | 'AbortIncompleteMultipartUpload';
storageClass?: string;
}
export interface LifecycleCondition {
age?: number;
createdBefore?: Date | string;
customTimeBefore?: Date | string;
daysSinceCustomTime?: number;
daysSinceNoncurrentTime?: number;
isLive?: boolean;
matchesPrefix?: string[];
matchesSuffix?: string[];
matchesStorageClass?: string[];
noncurrentTimeBefore?: Date | string;
numNewerVersions?: number;
}
export interface LifecycleRule {
action: LifecycleAction;
condition: LifecycleCondition;
}
export interface LifecycleCondition {
age?: number;
createdBefore?: Date | string;
customTimeBefore?: Date | string;
daysSinceCustomTime?: number;
daysSinceNoncurrentTime?: number;
isLive?: boolean;
matchesPrefix?: string[];
matchesSuffix?: string[];
matchesStorageClass?: string[];
noncurrentTimeBefore?: Date | string;
numNewerVersions?: number;
}
export interface LifecycleRule {
action: LifecycleAction;
condition: LifecycleCondition;
}
export interface EnableLoggingOptions extends PreconditionOptions {
bucket?: string | Bucket;
prefix: string;
}
export interface GetFilesOptions {
autoPaginate?: boolean;
delimiter?: string;
endOffset?: string;
includeFoldersAsPrefixes?: boolean;
includeTrailingDelimiter?: boolean;
prefix?: string;
matchGlob?: string;
maxApiCalls?: number;
maxResults?: number;
pageToken?: string;
softDeleted?: boolean;
startOffset?: string;
userProject?: string;
versions?: boolean;
fields?: string;
}
export interface CombineOptions extends PreconditionOptions {
kmsKeyName?: string;
userProject?: string;
}
export interface CombineCallback {
(err: Error | null, newFile: File | null, apiResponse: unknown): void;
}
export type CombineResponse = [File, unknown];
export interface CreateChannelConfig extends WatchAllOptions {
address: string;
}
export interface CreateChannelOptions {
userProject?: string;
}
export type CreateChannelResponse = [Channel, unknown];
export interface CreateChannelCallback {
(err: Error | null, channel: Channel | null, apiResponse: unknown): void;
}
export interface CreateNotificationOptions {
customAttributes?: {
[key: string]: string;
};
eventTypes?: string[];
objectNamePrefix?: string;
payloadFormat?: string;
userProject?: string;
}
export interface CreateNotificationCallback {
(err: Error | null, notification: Notification | null, apiResponse: unknown): void;
}
export type CreateNotificationResponse = [Notification, unknown];
export interface DeleteBucketOptions {
ignoreNotFound?: boolean;
userProject?: string;
}
export type DeleteBucketResponse = [unknown];
export interface DeleteBucketCallback extends DeleteCallback {
(err: Error | null, apiResponse: unknown): void;
}
export interface DeleteFilesOptions extends GetFilesOptions, PreconditionOptions {
force?: boolean;
}
export interface DeleteFilesCallback {
(err: Error | Error[] | null, apiResponse?: object): void;
}
export type DeleteLabelsResponse = [unknown];
export type DeleteLabelsCallback = SetLabelsCallback;
export type DeleteLabelsOptions = PreconditionOptions;
export type DisableRequesterPaysOptions = PreconditionOptions;
export type DisableRequesterPaysResponse = [unknown];
export interface DisableRequesterPaysCallback {
(err?: Error | null, apiResponse?: object): void;
}
export type EnableRequesterPaysResponse = [unknown];
export interface EnableRequesterPaysCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type EnableRequesterPaysOptions = PreconditionOptions;
export interface BucketExistsOptions extends GetConfig {
userProject?: string;
}
export type BucketExistsResponse = [boolean];
export type BucketExistsCallback = ExistsCallback;
export interface GetBucketOptions extends GetConfig {
userProject?: string;
}
export type GetBucketResponse = [Bucket, unknown];
export interface GetBucketCallback {
(err: ApiError | null, bucket: Bucket | null, apiResponse: unknown): void;
}
export interface GetLabelsOptions {
userProject?: string;
}
export type GetLabelsResponse = [unknown];
export interface GetLabelsCallback {
(err: Error | null, labels: object | null): void;
}
export interface RestoreOptions {
generation: string;
projection?: 'full' | 'noAcl';
}
export interface BucketMetadata extends BaseMetadata {
acl?: AclMetadata[] | null;
autoclass?: {
enabled?: boolean;
toggleTime?: string;
terminalStorageClass?: string;
terminalStorageClassUpdateTime?: string;
};
billing?: {
requesterPays?: boolean;
};
cors?: Cors[];
customPlacementConfig?: {
dataLocations?: string[];
};
defaultEventBasedHold?: boolean;
defaultObjectAcl?: AclMetadata[];
encryption?: {
defaultKmsKeyName?: string;
} | null;
hierarchicalNamespace?: {
enabled?: boolean;
};
iamConfiguration?: {
publicAccessPrevention?: string;
uniformBucketLevelAccess?: {
enabled?: boolean;
lockedTime?: string;
};
};
labels?: {
[key: string]: string | null;
};
lifecycle?: {
rule?: LifecycleRule[];
} | null;
location?: string;
locationType?: string;
logging?: {
logBucket?: string;
logObjectPrefix?: string;
};
generation?: string;
metageneration?: string;
name?: string;
objectRetention?: {
mode?: string;
};
owner?: {
entity?: string;
entityId?: string;
};
projectNumber?: string | number;
retentionPolicy?: {
effectiveTime?: string;
isLocked?: boolean;
retentionPeriod?: string | number;
} | null;
rpo?: string;
softDeleteTime?: string;
hardDeleteTime?: string;
softDeletePolicy?: {
retentionDurationSeconds?: string | number;
readonly effectiveTime?: string;
};
storageClass?: string;
timeCreated?: string;
updated?: string;
versioning?: {
enabled?: boolean;
};
website?: {
mainPageSuffix?: string;
notFoundPage?: string;
};
}
export type GetBucketMetadataResponse = [BucketMetadata, unknown];
export interface GetBucketMetadataCallback {
(err: ApiError | null, metadata: BucketMetadata | null, apiResponse: unknown): void;
}
export interface GetBucketMetadataOptions {
userProject?: string;
}
export interface GetBucketSignedUrlConfig extends Pick<SignerGetSignedUrlConfig, 'host' | 'signingEndpoint'> {
action: 'list';
version?: 'v2' | 'v4';
cname?: string;
virtualHostedStyle?: boolean;
expires: string | number | Date;
extensionHeaders?: http.OutgoingHttpHeaders;
queryParams?: Query;
}
export declare enum BucketActionToHTTPMethod {
list = "GET"
}
export declare enum AvailableServiceObjectMethods {
setMetadata = 0,
delete = 1
}
export interface GetNotificationsOptions {
userProject?: string;
}
export interface GetNotificationsCallback {
(err: Error | null, notifications: Notification[] | null, apiResponse: unknown): void;
}
export type GetNotificationsResponse = [Notification[], unknown];
export interface MakeBucketPrivateOptions {
includeFiles?: boolean;
force?: boolean;
metadata?: BucketMetadata;
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type MakeBucketPrivateResponse = [File[]];
export interface MakeBucketPrivateCallback {
(err?: Error | null, files?: File[]): void;
}
export interface MakeBucketPublicOptions {
includeFiles?: boolean;
force?: boolean;
}
export interface MakeBucketPublicCallback {
(err?: Error | null, files?: File[]): void;
}
export type MakeBucketPublicResponse = [File[]];
export interface SetBucketMetadataOptions extends PreconditionOptions {
userProject?: string;
}
export type SetBucketMetadataResponse = [BucketMetadata];
export interface SetBucketMetadataCallback {
(err?: Error | null, metadata?: BucketMetadata): void;
}
export interface BucketLockCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type BucketLockResponse = [unknown];
export interface Labels {
[key: string]: string;
}
export interface SetLabelsOptions extends PreconditionOptions {
userProject?: string;
}
export type SetLabelsResponse = [unknown];
export interface SetLabelsCallback {
(err?: Error | null, metadata?: unknown): void;
}
export interface SetBucketStorageClassOptions extends PreconditionOptions {
userProject?: string;
}
export interface SetBucketStorageClassCallback {
(err?: Error | null): void;
}
export type UploadResponse = [File, unknown];
export interface UploadCallback {
(err: Error | null, file?: File | null, apiResponse?: unknown): void;
}
export interface UploadOptions extends CreateResumableUploadOptions, CreateWriteStreamOptions {
destination?: string | File;
encryptionKey?: string | Buffer;
kmsKeyName?: string;
onUploadProgress?: (progressEvent: any) => void;
}
export interface MakeAllFilesPublicPrivateOptions {
force?: boolean;
private?: boolean;
public?: boolean;
userProject?: string;
}
interface MakeAllFilesPublicPrivateCallback {
(err?: Error | Error[] | null, files?: File[]): void;
}
type MakeAllFilesPublicPrivateResponse = [File[]];
export declare enum BucketExceptionMessages {
PROVIDE_SOURCE_FILE = "You must provide at least one source file.",
DESTINATION_FILE_NOT_SPECIFIED = "A destination file must be specified.",
CHANNEL_ID_REQUIRED = "An ID is required to create a channel.",
TOPIC_NAME_REQUIRED = "A valid topic name is required.",
CONFIGURATION_OBJECT_PREFIX_REQUIRED = "A configuration object with a prefix is required.",
SPECIFY_FILE_NAME = "A file name must be specified.",
METAGENERATION_NOT_PROVIDED = "A metageneration must be provided.",
SUPPLY_NOTIFICATION_ID = "You must supply a notification ID."
}
/**
* @callback Crc32cGeneratorToStringCallback
* A method returning the CRC32C as a base64-encoded string.
*
* @returns {string}
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
/**
* @callback Crc32cGeneratorValidateCallback
* A method validating a base64-encoded CRC32C string.
*
* @param {string} [value] base64-encoded CRC32C string to validate
* @returns {boolean}
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
**/
/**
* @callback Crc32cGeneratorUpdateCallback
* A method for passing `Buffer`s for CRC32C generation.
*
* @param {Buffer} [data] data to update CRC32C value with
* @returns {undefined}
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
**/
/**
* @typedef {object} CRC32CValidator
* @property {Crc32cGeneratorToStringCallback}
* @property {Crc32cGeneratorValidateCallback}
* @property {Crc32cGeneratorUpdateCallback}
*/
/**
* A function that generates a CRC32C Validator. Defaults to {@link CRC32C}
*
* @name Bucket#crc32cGenerator
* @type {CRC32CValidator}
*/
/**
* Get and set IAM policies for your bucket.
*
* @name Bucket#iam
* @mixes Iam
*
* See {@link https://cloud.google.com/storage/docs/access-control/iam#short_title_iam_management| Cloud Storage IAM Management}
* See {@link https://cloud.google.com/iam/docs/granting-changing-revoking-access| Granting, Changing, and Revoking Access}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
*
* //-
* // Get the IAM policy for your bucket.
* //-
* bucket.iam.getPolicy(function(err, policy) {
* console.log(policy);
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.getPolicy().then(function(data) {
* const policy = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_view_bucket_iam_members
* Example of retrieving a bucket's IAM policy:
*
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_add_bucket_iam_member
* Example of adding to a bucket's IAM policy:
*
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_remove_bucket_iam_member
* Example of removing from a bucket's IAM policy:
*/
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against
* an object or bucket (for example, `READ` or `WRITE`); the entity defines
* who the permission applies to (for example, a specific user or group of
* users).
*
* The `acl` object on a Bucket instance provides methods to get you a list of
* the ACLs defined on your bucket, as well as set, update, and delete them.
*
* Buckets also have
* {@link https://cloud.google.com/storage/docs/access-control/lists#default| default ACLs}
* for all created files. Default ACLs specify permissions that all new
* objects added to the bucket will inherit by default. You can add, delete,
* get, and update entities and permissions for these as well with
* {@link Bucket#acl.default}.
*
* See {@link http://goo.gl/6qBBPO| About Access Control Lists}
* See {@link https://cloud.google.com/storage/docs/access-control/lists#default| Default ACLs}
*
* @name Bucket#acl
* @mixes Acl
* @property {Acl} default Cloud Storage Buckets have
* {@link https://cloud.google.com/storage/docs/access-control/lists#default| default ACLs}
* for all created files. You can add, delete, get, and update entities and
* permissions for these as well. The method signatures and examples are all
* the same, after only prefixing the method call with `default`.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Make a bucket's contents publicly readable.
* //-
* const myBucket = storage.bucket('my-bucket');
*
* const options = {
* entity: 'allUsers',
* role: storage.acl.READER_ROLE
* };
*
* myBucket.acl.add(options, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* myBucket.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl
* Example of printing a bucket's ACL:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_print_bucket_acl_for_user
* Example of printing a bucket's ACL for a specific user:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_owner
* Example of adding an owner to a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_owner
* Example of removing an owner from a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_add_bucket_default_owner
* Example of adding a default owner to a bucket:
*
* @example <caption>include:samples/acl.js</caption>
* region_tag:storage_remove_bucket_default_owner
* Example of removing a default owner from a bucket:
*/
/**
* The API-formatted resource description of the bucket.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name Bucket#metadata
* @type {object}
*/
/**
* The bucket's name.
* @name Bucket#name
* @type {string}
*/
/**
* Get {@link File} objects for the files currently in the bucket as a
* readable object stream.
*
* @method Bucket#getFilesStream
* @param {GetFilesOptions} [query] Query object for listing files.
* @returns {ReadableStream} A readable stream that emits {@link File} instances.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
*
* bucket.getFilesStream()
* .on('error', console.error)
* .on('data', function(file) {
* // file is a File object.
* })
* .on('end', function() {
* // All files retrieved.
* });
*
* //-
* // If you anticipate many results, you can end a stream early to prevent
* // unnecessary processing and API requests.
* //-
* bucket.getFilesStream()
* .on('data', function(file) {
* this.end();
* });
*
* //-
* // If you're filtering files with a delimiter, you should use
* // {@link Bucket#getFiles} and set `autoPaginate: false` in order to
* // preserve the `apiResponse` argument.
* //-
* const prefixes = [];
*
* function callback(err, files, nextQuery, apiResponse) {
* prefixes = prefixes.concat(apiResponse.prefixes);
*
* if (nextQuery) {
* bucket.getFiles(nextQuery, callback);
* } else {
* // prefixes = The finished array of prefixes.
* }
* }
*
* bucket.getFiles({
* autoPaginate: false,
* delimiter: '/'
* }, callback);
* ```
*/
/**
* Create a Bucket object to interact with a Cloud Storage bucket.
*
* @class
* @hideconstructor
*
* @param {Storage} storage A {@link Storage} instance.
* @param {string} name The name of the bucket.
* @param {object} [options] Configuration object.
* @param {string} [options.userProject] User project.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* ```
*/
declare class Bucket extends ServiceObject<Bucket, BucketMetadata> {
name: string;
/**
* A reference to the {@link Storage} associated with this {@link Bucket}
* instance.
* @name Bucket#storage
* @type {Storage}
*/
storage: Storage;
/**
* A user project to apply to each request from this bucket.
* @name Bucket#userProject
* @type {string}
*/
userProject?: string;
acl: Acl;
iam: Iam;
crc32cGenerator: CRC32CValidatorGenerator;
getFilesStream(query?: GetFilesOptions): Readable;
signer?: URLSigner;
private instanceRetryValue?;
instancePreconditionOpts?: PreconditionOptions;
constructor(storage: Storage, name: string, options?: BucketOptions);
/**
* The bucket's Cloud Storage URI (`gs://`)
*
* @example
* ```ts
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* // `gs://my-bucket`
* const href = bucket.cloudStorageURI.href;
* ```
*/
get cloudStorageURI(): URL;
addLifecycleRule(rule: LifecycleRule | LifecycleRule[], options?: AddLifecycleRuleOptions): Promise<SetBucketMetadataResponse>;
addLifecycleRule(rule: LifecycleRule | LifecycleRule[], options: AddLifecycleRuleOptions, callback: SetBucketMetadataCallback): void;
addLifecycleRule(rule: LifecycleRule | LifecycleRule[], callback: SetBucketMetadataCallback): void;
combine(sources: string[] | File[], destination: string | File, options?: CombineOptions): Promise<CombineResponse>;
combine(sources: string[] | File[], destination: string | File, options: CombineOptions, callback: CombineCallback): void;
combine(sources: string[] | File[], destination: string | File, callback: CombineCallback): void;
createChannel(id: string, config: CreateChannelConfig, options?: CreateChannelOptions): Promise<CreateChannelResponse>;
createChannel(id: string, config: CreateChannelConfig, callback: CreateChannelCallback): void;
createChannel(id: string, config: CreateChannelConfig, options: CreateChannelOptions, callback: CreateChannelCallback): void;
createNotification(topic: string, options?: CreateNotificationOptions): Promise<CreateNotificationResponse>;
createNotification(topic: string, options: CreateNotificationOptions, callback: CreateNotificationCallback): void;
createNotification(topic: string, callback: CreateNotificationCallback): void;
deleteFiles(query?: DeleteFilesOptions): Promise<void>;
deleteFiles(callback: DeleteFilesCallback): void;
deleteFiles(query: DeleteFilesOptions, callback: DeleteFilesCallback): void;
deleteLabels(labels?: string | string[]): Promise<DeleteLabelsResponse>;
deleteLabels(options: DeleteLabelsOptions): Promise<DeleteLabelsResponse>;
deleteLabels(callback: DeleteLabelsCallback): void;
deleteLabels(labels: string | string[], options: DeleteLabelsOptions): Promise<DeleteLabelsResponse>;
deleteLabels(labels: string | string[], callback: DeleteLabelsCallback): void;
deleteLabels(labels: string | string[], options: DeleteLabelsOptions, callback: DeleteLabelsCallback): void;
disableRequesterPays(options?: DisableRequesterPaysOptions): Promise<DisableRequesterPaysResponse>;
disableRequesterPays(callback: DisableRequesterPaysCallback): void;
disableRequesterPays(options: DisableRequesterPaysOptions, callback: DisableRequesterPaysCallback): void;
enableLogging(config: EnableLoggingOptions): Promise<SetBucketMetadataResponse>;
enableLogging(config: EnableLoggingOptions, callback: SetBucketMetadataCallback): void;
enableRequesterPays(options?: EnableRequesterPaysOptions): Promise<EnableRequesterPaysResponse>;
enableRequesterPays(callback: EnableRequesterPaysCallback): void;
enableRequesterPays(options: EnableRequesterPaysOptions, callback: EnableRequesterPaysCallback): void;
/**
* Create a {@link File} object. See {@link File} to see how to handle
* the different use cases you may have.
*
* @param {string} name The name of the file in this bucket.
* @param {FileOptions} [options] Configuration options.
* @param {string|number} [options.generation] Only use a specific revision of
* this file.
* @param {string} [options.encryptionKey] A custom encryption key. See
* {@link https://cloud.google.com/storage/docs/encryption#customer-supplied| Customer-supplied Encryption Keys}.
* @param {string} [options.kmsKeyName] The name of the Cloud KMS key that will
* be used to encrypt the object. Must be in the format:
* `projects/my-project/locations/location/keyRings/my-kr/cryptoKeys/my-key`.
* KMS key ring must use the same location as the bucket.
* @param {string} [options.userProject] The ID of the project which will be
* billed for all requests made from File object.
* @returns {File}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* const file = bucket.file('my-existing-file.png');
* ```
*/
file(name: string, options?: FileOptions): File;
getFiles(query?: GetFilesOptions): Promise<GetFilesResponse>;
getFiles(query: GetFilesOptions, callback: GetFilesCallback): void;
getFiles(callback: GetFilesCallback): void;
getLabels(options?: GetLabelsOptions): Promise<GetLabelsResponse>;
getLabels(callback: GetLabelsCallback): void;
getLabels(options: GetLabelsOptions, callback: GetLabelsCallback): void;
getNotifications(options?: GetNotificationsOptions): Promise<GetNotificationsResponse>;
getNotifications(callback: GetNotificationsCallback): void;
getNotifications(options: GetNotificationsOptions, callback: GetNotificationsCallback): void;
getSignedUrl(cfg: GetBucketSignedUrlConfig): Promise<GetSignedUrlResponse>;
getSignedUrl(cfg: GetBucketSignedUrlConfig, callback: GetSignedUrlCallback): void;
lock(metageneration: number | string): Promise<BucketLockResponse>;
lock(metageneration: number | string, callback: BucketLockCallback): void;
/**
* @typedef {object} RestoreOptions Options for Bucket#restore(). See an
* {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/restore#resource| Object resource}.
* @param {number} [generation] If present, selects a specific revision of this object.
* @param {string} [projection] Specifies the set of properties to return. If used, must be 'full' or 'noAcl'.
*/
/**
* Restores a soft-deleted bucket
* @param {RestoreOptions} options Restore options.
* @returns {Promise<Bucket>}
*/
restore(options: RestoreOptions): Promise<Bucket>;
makePrivate(options?: MakeBucketPrivateOptions): Promise<MakeBucketPrivateResponse>;
makePrivate(callback: MakeBucketPrivateCallback): void;
makePrivate(options: MakeBucketPrivateOptions, callback: MakeBucketPrivateCallback): void;
makePublic(options?: MakeBucketPublicOptions): Promise<MakeBucketPublicResponse>;
makePublic(callback: MakeBucketPublicCallback): void;
makePublic(options: MakeBucketPublicOptions, callback: MakeBucketPublicCallback): void;
/**
* Get a reference to a Cloud Pub/Sub Notification.
*
* @param {string} id ID of notification.
* @returns {Notification}
* @see Notification
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const notification = bucket.notification('1');
* ```
*/
notification(id: string): Notification;
removeRetentionPeriod(options?: SetBucketMetadataOptions): Promise<SetBucketMetadataResponse>;
removeRetentionPeriod(callback: SetBucketMetadataCallback): void;
removeRetentionPeriod(options: SetBucketMetadataOptions, callback: SetBucketMetadataCallback): void;
request(reqOpts: DecorateRequestOptions): Promise<RequestResponse>;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
setLabels(labels: Labels, options?: SetLabelsOptions): Promise<SetLabelsResponse>;
setLabels(labels: Labels, callback: SetLabelsCallback): void;
setLabels(labels: Labels, options: SetLabelsOptions, callback: SetLabelsCallback): void;
setMetadata(metadata: BucketMetadata, options?: SetMetadataOptions): Promise<SetMetadataResponse<BucketMetadata>>;
setMetadata(metadata: BucketMetadata, callback: MetadataCallback<BucketMetadata>): void;
setMetadata(metadata: BucketMetadata, options: SetMetadataOptions, callback: MetadataCallback<BucketMetadata>): void;
setRetentionPeriod(duration: number, options?: SetBucketMetadataOptions): Promise<SetBucketMetadataResponse>;
setRetentionPeriod(duration: number, callback: SetBucketMetadataCallback): void;
setRetentionPeriod(duration: number, options: SetBucketMetadataOptions, callback: SetBucketMetadataCallback): void;
setCorsConfiguration(corsConfiguration: Cors[], options?: SetBucketMetadataOptions): Promise<SetBucketMetadataResponse>;
setCorsConfiguration(corsConfiguration: Cors[], callback: SetBucketMetadataCallback): void;
setCorsConfiguration(corsConfiguration: Cors[], options: SetBucketMetadataOptions, callback: SetBucketMetadataCallback): void;
setStorageClass(storageClass: string, options?: SetBucketStorageClassOptions): Promise<SetBucketMetadataResponse>;
setStorageClass(storageClass: string, callback: SetBucketStorageClassCallback): void;
setStorageClass(storageClass: string, options: SetBucketStorageClassOptions, callback: SetBucketStorageClassCallback): void;
/**
* Set a user project to be billed for all requests made from this Bucket
* object and any files referenced from this Bucket object.
*
* @param {string} userProject The user project.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
*
* bucket.setUserProject('grape-spaceship-123');
* ```
*/
setUserProject(userProject: string): void;
upload(pathString: string, options?: UploadOptions): Promise<UploadResponse>;
upload(pathString: string, options: UploadOptions, callback: UploadCallback): void;
upload(pathString: string, callback: UploadCallback): void;
makeAllFilesPublicPrivate_(options?: MakeAllFilesPublicPrivateOptions): Promise<MakeAllFilesPublicPrivateResponse>;
makeAllFilesPublicPrivate_(callback: MakeAllFilesPublicPrivateCallback): void;
makeAllFilesPublicPrivate_(options: MakeAllFilesPublicPrivateOptions, callback: MakeAllFilesPublicPrivateCallback): void;
getId(): string;
disableAutoRetryConditionallyIdempotent_(coreOpts: any, methodType: AvailableServiceObjectMethods, localPreconditionOptions?: PreconditionOptions): void;
}
/**
* Reference to the {@link Bucket} class.
* @name module:@google-cloud/storage.Bucket
* @see Bucket
*/
export { Bucket };

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,33 @@
import { BaseMetadata, ServiceObject } from './nodejs-common/index.js';
import { Storage } from './storage.js';
export interface StopCallback {
(err: Error | null, apiResponse?: unknown): void;
}
/**
* Create a channel object to interact with a Cloud Storage channel.
*
* See {@link https://cloud.google.com/storage/docs/object-change-notification| Object Change Notification}
*
* @class
*
* @param {string} id The ID of the channel.
* @param {string} resourceId The resource ID of the channel.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* ```
*/
declare class Channel extends ServiceObject<Channel, BaseMetadata> {
constructor(storage: Storage, id: string, resourceId: string);
stop(): Promise<unknown>;
stop(callback: StopCallback): void;
}
/**
* Reference to the {@link Channel} class.
* @name module:@google-cloud/storage.Channel
* @see Channel
*/
export { Channel };

View File

@@ -0,0 +1,106 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ServiceObject, util } from './nodejs-common/index.js';
import { promisifyAll } from '@google-cloud/promisify';
/**
* Create a channel object to interact with a Cloud Storage channel.
*
* See {@link https://cloud.google.com/storage/docs/object-change-notification| Object Change Notification}
*
* @class
*
* @param {string} id The ID of the channel.
* @param {string} resourceId The resource ID of the channel.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* ```
*/
class Channel extends ServiceObject {
constructor(storage, id, resourceId) {
const config = {
parent: storage,
baseUrl: '/channels',
// An ID shouldn't be included in the API requests.
// RE:
// https://github.com/GoogleCloudPlatform/google-cloud-node/issues/1145
id: '',
methods: {
// Only need `request`.
},
};
super(config);
this.metadata.id = id;
this.metadata.resourceId = resourceId;
}
/**
* @typedef {array} StopResponse
* @property {object} 0 The full API response.
*/
/**
* @callback StopCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
/**
* Stop this channel.
*
* @param {StopCallback} [callback] Callback function.
* @returns {Promise<StopResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* channel.stop(function(err, apiResponse) {
* if (!err) {
* // Channel stopped successfully.
* }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* channel.stop().then(function(data) {
* const apiResponse = data[0];
* });
* ```
*/
stop(callback) {
callback = callback || util.noop;
this.request({
method: 'POST',
uri: '/stop',
json: this.metadata,
}, (err, apiResponse) => {
callback(err, apiResponse);
});
}
}
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
promisifyAll(Channel);
/**
* Reference to the {@link Channel} class.
* @name module:@google-cloud/storage.Channel
* @see Channel
*/
export { Channel };

View File

@@ -0,0 +1,143 @@
import { PathLike } from 'fs';
/**
* Ported from {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc#L16-L59 github.com/google/crc32c}
*/
declare const CRC32C_EXTENSIONS: readonly [0, 4067132163, 3778769143, 324072436, 3348797215, 904991772, 648144872, 3570033899, 2329499855, 2024987596, 1809983544, 2575936315, 1296289744, 3207089363, 2893594407, 1578318884, 274646895, 3795141740, 4049975192, 51262619, 3619967088, 632279923, 922689671, 3298075524, 2592579488, 1760304291, 2075979607, 2312596564, 1562183871, 2943781820, 3156637768, 1313733451, 549293790, 3537243613, 3246849577, 871202090, 3878099393, 357341890, 102525238, 4101499445, 2858735121, 1477399826, 1264559846, 3107202533, 1845379342, 2677391885, 2361733625, 2125378298, 820201905, 3263744690, 3520608582, 598981189, 4151959214, 85089709, 373468761, 3827903834, 3124367742, 1213305469, 1526817161, 2842354314, 2107672161, 2412447074, 2627466902, 1861252501, 1098587580, 3004210879, 2688576843, 1378610760, 2262928035, 1955203488, 1742404180, 2511436119, 3416409459, 969524848, 714683780, 3639785095, 205050476, 4266873199, 3976438427, 526918040, 1361435347, 2739821008, 2954799652, 1114974503, 2529119692, 1691668175, 2005155131, 2247081528, 3690758684, 697762079, 986182379, 3366744552, 476452099, 3993867776, 4250756596, 255256311, 1640403810, 2477592673, 2164122517, 1922457750, 2791048317, 1412925310, 1197962378, 3037525897, 3944729517, 427051182, 170179418, 4165941337, 746937522, 3740196785, 3451792453, 1070968646, 1905808397, 2213795598, 2426610938, 1657317369, 3053634322, 1147748369, 1463399397, 2773627110, 4215344322, 153784257, 444234805, 3893493558, 1021025245, 3467647198, 3722505002, 797665321, 2197175160, 1889384571, 1674398607, 2443626636, 1164749927, 3070701412, 2757221520, 1446797203, 137323447, 4198817972, 3910406976, 461344835, 3484808360, 1037989803, 781091935, 3705997148, 2460548119, 1623424788, 1939049696, 2180517859, 1429367560, 2807687179, 3020495871, 1180866812, 410100952, 3927582683, 4182430767, 186734380, 3756733383, 763408580, 1053836080, 3434856499, 2722870694, 1344288421, 1131464017, 2971354706, 1708204729, 2545590714, 2229949006, 1988219213, 680717673, 3673779818, 3383336350, 1002577565, 4010310262, 493091189, 238226049, 4233660802, 2987750089, 1082061258, 1395524158, 2705686845, 1972364758, 2279892693, 2494862625, 1725896226, 952904198, 3399985413, 3656866545, 731699698, 4283874585, 222117402, 510512622, 3959836397, 3280807620, 837199303, 582374963, 3504198960, 68661723, 4135334616, 3844915500, 390545967, 1230274059, 3141532936, 2825850620, 1510247935, 2395924756, 2091215383, 1878366691, 2644384480, 3553878443, 565732008, 854102364, 3229815391, 340358836, 3861050807, 4117890627, 119113024, 1493875044, 2875275879, 3090270611, 1247431312, 2660249211, 1828433272, 2141937292, 2378227087, 3811616794, 291187481, 34330861, 4032846830, 615137029, 3603020806, 3314634738, 939183345, 1776939221, 2609017814, 2295496738, 2058945313, 2926798794, 1545135305, 1330124605, 3173225534, 4084100981, 17165430, 307568514, 3762199681, 888469610, 3332340585, 3587147933, 665062302, 2042050490, 2346497209, 2559330125, 1793573966, 3190661285, 1279665062, 1595330642, 2910671697];
declare const CRC32C_EXTENSION_TABLE: Int32Array<ArrayBuffer>;
/** An interface for CRC32C hashing and validation */
interface CRC32CValidator {
/**
* A method returning the CRC32C as a base64-encoded string.
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
toString: () => string;
/**
* A method validating a base64-encoded CRC32C string.
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
*/
validate: (value: string) => boolean;
/**
* A method for passing `Buffer`s for CRC32C generation.
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
*/
update: (data: Buffer) => void;
}
/** A function that generates a CRC32C Validator */
interface CRC32CValidatorGenerator {
/** Should return a new, ready-to-use `CRC32CValidator` */
(): CRC32CValidator;
}
declare const CRC32C_DEFAULT_VALIDATOR_GENERATOR: CRC32CValidatorGenerator;
declare const CRC32C_EXCEPTION_MESSAGES: {
readonly INVALID_INIT_BASE64_RANGE: (l: number) => string;
readonly INVALID_INIT_BUFFER_LENGTH: (l: number) => string;
readonly INVALID_INIT_INTEGER: (l: number) => string;
};
declare class CRC32C implements CRC32CValidator {
#private;
/**
* Constructs a new `CRC32C` object.
*
* Reconstruction is recommended via the `CRC32C.from` static method.
*
* @param initialValue An initial CRC32C value - a signed 32-bit integer.
*/
constructor(initialValue?: number);
/**
* Calculates a CRC32C from a provided buffer.
*
* Implementation inspired from:
* - {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc github.com/google/crc32c}
* - {@link https://github.com/googleapis/python-crc32c/blob/a595e758c08df445a99c3bf132ee8e80a3ec4308/src/google_crc32c/python.py github.com/googleapis/python-crc32c}
* - {@link https://github.com/googleapis/java-storage/pull/1376/files github.com/googleapis/java-storage}
*
* @param data The `Buffer` to generate the CRC32C from
*/
update(data: Buffer): void;
/**
* Validates a provided input to the current CRC32C value.
*
* @param input A Buffer, `CRC32C`-compatible object, base64-encoded data (string), or signed 32-bit integer
*/
validate(input: Buffer | CRC32CValidator | string | number): boolean;
/**
* Returns a `Buffer` representation of the CRC32C value
*/
toBuffer(): Buffer;
/**
* Returns a JSON-compatible, base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify `JSON#stringify`}
*/
toJSON(): string;
/**
* Returns a base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/toString `Object#toString`}
*/
toString(): string;
/**
* Returns the `number` representation of the CRC32C value as a signed 32-bit integer
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/valueOf `Object#valueOf`}
*/
valueOf(): number;
static readonly CRC32C_EXTENSIONS: readonly [0, 4067132163, 3778769143, 324072436, 3348797215, 904991772, 648144872, 3570033899, 2329499855, 2024987596, 1809983544, 2575936315, 1296289744, 3207089363, 2893594407, 1578318884, 274646895, 3795141740, 4049975192, 51262619, 3619967088, 632279923, 922689671, 3298075524, 2592579488, 1760304291, 2075979607, 2312596564, 1562183871, 2943781820, 3156637768, 1313733451, 549293790, 3537243613, 3246849577, 871202090, 3878099393, 357341890, 102525238, 4101499445, 2858735121, 1477399826, 1264559846, 3107202533, 1845379342, 2677391885, 2361733625, 2125378298, 820201905, 3263744690, 3520608582, 598981189, 4151959214, 85089709, 373468761, 3827903834, 3124367742, 1213305469, 1526817161, 2842354314, 2107672161, 2412447074, 2627466902, 1861252501, 1098587580, 3004210879, 2688576843, 1378610760, 2262928035, 1955203488, 1742404180, 2511436119, 3416409459, 969524848, 714683780, 3639785095, 205050476, 4266873199, 3976438427, 526918040, 1361435347, 2739821008, 2954799652, 1114974503, 2529119692, 1691668175, 2005155131, 2247081528, 3690758684, 697762079, 986182379, 3366744552, 476452099, 3993867776, 4250756596, 255256311, 1640403810, 2477592673, 2164122517, 1922457750, 2791048317, 1412925310, 1197962378, 3037525897, 3944729517, 427051182, 170179418, 4165941337, 746937522, 3740196785, 3451792453, 1070968646, 1905808397, 2213795598, 2426610938, 1657317369, 3053634322, 1147748369, 1463399397, 2773627110, 4215344322, 153784257, 444234805, 3893493558, 1021025245, 3467647198, 3722505002, 797665321, 2197175160, 1889384571, 1674398607, 2443626636, 1164749927, 3070701412, 2757221520, 1446797203, 137323447, 4198817972, 3910406976, 461344835, 3484808360, 1037989803, 781091935, 3705997148, 2460548119, 1623424788, 1939049696, 2180517859, 1429367560, 2807687179, 3020495871, 1180866812, 410100952, 3927582683, 4182430767, 186734380, 3756733383, 763408580, 1053836080, 3434856499, 2722870694, 1344288421, 1131464017, 2971354706, 1708204729, 2545590714, 2229949006, 1988219213, 680717673, 3673779818, 3383336350, 1002577565, 4010310262, 493091189, 238226049, 4233660802, 2987750089, 1082061258, 1395524158, 2705686845, 1972364758, 2279892693, 2494862625, 1725896226, 952904198, 3399985413, 3656866545, 731699698, 4283874585, 222117402, 510512622, 3959836397, 3280807620, 837199303, 582374963, 3504198960, 68661723, 4135334616, 3844915500, 390545967, 1230274059, 3141532936, 2825850620, 1510247935, 2395924756, 2091215383, 1878366691, 2644384480, 3553878443, 565732008, 854102364, 3229815391, 340358836, 3861050807, 4117890627, 119113024, 1493875044, 2875275879, 3090270611, 1247431312, 2660249211, 1828433272, 2141937292, 2378227087, 3811616794, 291187481, 34330861, 4032846830, 615137029, 3603020806, 3314634738, 939183345, 1776939221, 2609017814, 2295496738, 2058945313, 2926798794, 1545135305, 1330124605, 3173225534, 4084100981, 17165430, 307568514, 3762199681, 888469610, 3332340585, 3587147933, 665062302, 2042050490, 2346497209, 2559330125, 1793573966, 3190661285, 1279665062, 1595330642, 2910671697];
static readonly CRC32C_EXTENSION_TABLE: Int32Array<ArrayBuffer>;
/**
* Generates a `CRC32C` from a compatible buffer format.
*
* @param value 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`
*/
private static fromBuffer;
static fromFile(file: PathLike): Promise<CRC32C>;
/**
* Generates a `CRC32C` from 4-byte base64-encoded data (string).
*
* @param value 4-byte base64-encoded data (string)
*/
private static fromString;
/**
* Generates a `CRC32C` from a safe, unsigned 32-bit integer.
*
* @param value an unsigned 32-bit integer
*/
private static fromNumber;
/**
* Generates a `CRC32C` from a variety of compatable types.
* Note: strings are treated as input, not as file paths to read from.
*
* @param value A number, 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`, or 4-byte base64-encoded data (string)
*/
static from(value: ArrayBuffer | ArrayBufferView | CRC32CValidator | string | number): CRC32C;
}
export { CRC32C, CRC32C_DEFAULT_VALIDATOR_GENERATOR, CRC32C_EXCEPTION_MESSAGES, CRC32C_EXTENSIONS, CRC32C_EXTENSION_TABLE, CRC32CValidator, CRC32CValidatorGenerator, };

View File

@@ -0,0 +1,254 @@
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _CRC32C_crc32c;
import { createReadStream } from 'fs';
/**
* Ported from {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc#L16-L59 github.com/google/crc32c}
*/
const CRC32C_EXTENSIONS = [
0x00000000, 0xf26b8303, 0xe13b70f7, 0x1350f3f4, 0xc79a971f, 0x35f1141c,
0x26a1e7e8, 0xd4ca64eb, 0x8ad958cf, 0x78b2dbcc, 0x6be22838, 0x9989ab3b,
0x4d43cfd0, 0xbf284cd3, 0xac78bf27, 0x5e133c24, 0x105ec76f, 0xe235446c,
0xf165b798, 0x030e349b, 0xd7c45070, 0x25afd373, 0x36ff2087, 0xc494a384,
0x9a879fa0, 0x68ec1ca3, 0x7bbcef57, 0x89d76c54, 0x5d1d08bf, 0xaf768bbc,
0xbc267848, 0x4e4dfb4b, 0x20bd8ede, 0xd2d60ddd, 0xc186fe29, 0x33ed7d2a,
0xe72719c1, 0x154c9ac2, 0x061c6936, 0xf477ea35, 0xaa64d611, 0x580f5512,
0x4b5fa6e6, 0xb93425e5, 0x6dfe410e, 0x9f95c20d, 0x8cc531f9, 0x7eaeb2fa,
0x30e349b1, 0xc288cab2, 0xd1d83946, 0x23b3ba45, 0xf779deae, 0x05125dad,
0x1642ae59, 0xe4292d5a, 0xba3a117e, 0x4851927d, 0x5b016189, 0xa96ae28a,
0x7da08661, 0x8fcb0562, 0x9c9bf696, 0x6ef07595, 0x417b1dbc, 0xb3109ebf,
0xa0406d4b, 0x522bee48, 0x86e18aa3, 0x748a09a0, 0x67dafa54, 0x95b17957,
0xcba24573, 0x39c9c670, 0x2a993584, 0xd8f2b687, 0x0c38d26c, 0xfe53516f,
0xed03a29b, 0x1f682198, 0x5125dad3, 0xa34e59d0, 0xb01eaa24, 0x42752927,
0x96bf4dcc, 0x64d4cecf, 0x77843d3b, 0x85efbe38, 0xdbfc821c, 0x2997011f,
0x3ac7f2eb, 0xc8ac71e8, 0x1c661503, 0xee0d9600, 0xfd5d65f4, 0x0f36e6f7,
0x61c69362, 0x93ad1061, 0x80fde395, 0x72966096, 0xa65c047d, 0x5437877e,
0x4767748a, 0xb50cf789, 0xeb1fcbad, 0x197448ae, 0x0a24bb5a, 0xf84f3859,
0x2c855cb2, 0xdeeedfb1, 0xcdbe2c45, 0x3fd5af46, 0x7198540d, 0x83f3d70e,
0x90a324fa, 0x62c8a7f9, 0xb602c312, 0x44694011, 0x5739b3e5, 0xa55230e6,
0xfb410cc2, 0x092a8fc1, 0x1a7a7c35, 0xe811ff36, 0x3cdb9bdd, 0xceb018de,
0xdde0eb2a, 0x2f8b6829, 0x82f63b78, 0x709db87b, 0x63cd4b8f, 0x91a6c88c,
0x456cac67, 0xb7072f64, 0xa457dc90, 0x563c5f93, 0x082f63b7, 0xfa44e0b4,
0xe9141340, 0x1b7f9043, 0xcfb5f4a8, 0x3dde77ab, 0x2e8e845f, 0xdce5075c,
0x92a8fc17, 0x60c37f14, 0x73938ce0, 0x81f80fe3, 0x55326b08, 0xa759e80b,
0xb4091bff, 0x466298fc, 0x1871a4d8, 0xea1a27db, 0xf94ad42f, 0x0b21572c,
0xdfeb33c7, 0x2d80b0c4, 0x3ed04330, 0xccbbc033, 0xa24bb5a6, 0x502036a5,
0x4370c551, 0xb11b4652, 0x65d122b9, 0x97baa1ba, 0x84ea524e, 0x7681d14d,
0x2892ed69, 0xdaf96e6a, 0xc9a99d9e, 0x3bc21e9d, 0xef087a76, 0x1d63f975,
0x0e330a81, 0xfc588982, 0xb21572c9, 0x407ef1ca, 0x532e023e, 0xa145813d,
0x758fe5d6, 0x87e466d5, 0x94b49521, 0x66df1622, 0x38cc2a06, 0xcaa7a905,
0xd9f75af1, 0x2b9cd9f2, 0xff56bd19, 0x0d3d3e1a, 0x1e6dcdee, 0xec064eed,
0xc38d26c4, 0x31e6a5c7, 0x22b65633, 0xd0ddd530, 0x0417b1db, 0xf67c32d8,
0xe52cc12c, 0x1747422f, 0x49547e0b, 0xbb3ffd08, 0xa86f0efc, 0x5a048dff,
0x8ecee914, 0x7ca56a17, 0x6ff599e3, 0x9d9e1ae0, 0xd3d3e1ab, 0x21b862a8,
0x32e8915c, 0xc083125f, 0x144976b4, 0xe622f5b7, 0xf5720643, 0x07198540,
0x590ab964, 0xab613a67, 0xb831c993, 0x4a5a4a90, 0x9e902e7b, 0x6cfbad78,
0x7fab5e8c, 0x8dc0dd8f, 0xe330a81a, 0x115b2b19, 0x020bd8ed, 0xf0605bee,
0x24aa3f05, 0xd6c1bc06, 0xc5914ff2, 0x37faccf1, 0x69e9f0d5, 0x9b8273d6,
0x88d28022, 0x7ab90321, 0xae7367ca, 0x5c18e4c9, 0x4f48173d, 0xbd23943e,
0xf36e6f75, 0x0105ec76, 0x12551f82, 0xe03e9c81, 0x34f4f86a, 0xc69f7b69,
0xd5cf889d, 0x27a40b9e, 0x79b737ba, 0x8bdcb4b9, 0x988c474d, 0x6ae7c44e,
0xbe2da0a5, 0x4c4623a6, 0x5f16d052, 0xad7d5351,
];
const CRC32C_EXTENSION_TABLE = new Int32Array(CRC32C_EXTENSIONS);
const CRC32C_DEFAULT_VALIDATOR_GENERATOR = () => new CRC32C();
const CRC32C_EXCEPTION_MESSAGES = {
INVALID_INIT_BASE64_RANGE: (l) => `base64-encoded data expected to equal 4 bytes, not ${l}`,
INVALID_INIT_BUFFER_LENGTH: (l) => `Buffer expected to equal 4 bytes, not ${l}`,
INVALID_INIT_INTEGER: (l) => `Number expected to be a safe, unsigned 32-bit integer, not ${l}`,
};
class CRC32C {
/**
* Constructs a new `CRC32C` object.
*
* Reconstruction is recommended via the `CRC32C.from` static method.
*
* @param initialValue An initial CRC32C value - a signed 32-bit integer.
*/
constructor(initialValue = 0) {
/** Current CRC32C value */
_CRC32C_crc32c.set(this, 0);
__classPrivateFieldSet(this, _CRC32C_crc32c, initialValue, "f");
}
/**
* Calculates a CRC32C from a provided buffer.
*
* Implementation inspired from:
* - {@link https://github.com/google/crc32c/blob/21fc8ef30415a635e7351ffa0e5d5367943d4a94/src/crc32c_portable.cc github.com/google/crc32c}
* - {@link https://github.com/googleapis/python-crc32c/blob/a595e758c08df445a99c3bf132ee8e80a3ec4308/src/google_crc32c/python.py github.com/googleapis/python-crc32c}
* - {@link https://github.com/googleapis/java-storage/pull/1376/files github.com/googleapis/java-storage}
*
* @param data The `Buffer` to generate the CRC32C from
*/
update(data) {
let current = __classPrivateFieldGet(this, _CRC32C_crc32c, "f") ^ 0xffffffff;
for (const d of data) {
const tablePoly = CRC32C.CRC32C_EXTENSION_TABLE[(d ^ current) & 0xff];
current = tablePoly ^ (current >>> 8);
}
__classPrivateFieldSet(this, _CRC32C_crc32c, current ^ 0xffffffff, "f");
}
/**
* Validates a provided input to the current CRC32C value.
*
* @param input A Buffer, `CRC32C`-compatible object, base64-encoded data (string), or signed 32-bit integer
*/
validate(input) {
if (typeof input === 'number') {
return input === __classPrivateFieldGet(this, _CRC32C_crc32c, "f");
}
else if (typeof input === 'string') {
return input === this.toString();
}
else if (Buffer.isBuffer(input)) {
return Buffer.compare(input, this.toBuffer()) === 0;
}
else {
// `CRC32C`-like object
return input.toString() === this.toString();
}
}
/**
* Returns a `Buffer` representation of the CRC32C value
*/
toBuffer() {
const buffer = Buffer.alloc(4);
buffer.writeInt32BE(__classPrivateFieldGet(this, _CRC32C_crc32c, "f"));
return buffer;
}
/**
* Returns a JSON-compatible, base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify `JSON#stringify`}
*/
toJSON() {
return this.toString();
}
/**
* Returns a base64-encoded representation of the CRC32C value.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/toString `Object#toString`}
*/
toString() {
return this.toBuffer().toString('base64');
}
/**
* Returns the `number` representation of the CRC32C value as a signed 32-bit integer
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/valueOf `Object#valueOf`}
*/
valueOf() {
return __classPrivateFieldGet(this, _CRC32C_crc32c, "f");
}
/**
* Generates a `CRC32C` from a compatible buffer format.
*
* @param value 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`
*/
static fromBuffer(value) {
let buffer;
if (Buffer.isBuffer(value)) {
buffer = value;
}
else if ('buffer' in value) {
// `ArrayBufferView`
buffer = Buffer.from(value.buffer);
}
else {
// `ArrayBuffer`
buffer = Buffer.from(value);
}
if (buffer.byteLength !== 4) {
throw new RangeError(CRC32C_EXCEPTION_MESSAGES.INVALID_INIT_BUFFER_LENGTH(buffer.byteLength));
}
return new CRC32C(buffer.readInt32BE());
}
static async fromFile(file) {
const crc32c = new CRC32C();
await new Promise((resolve, reject) => {
createReadStream(file)
.on('data', (d) => {
if (typeof d === 'string') {
crc32c.update(Buffer.from(d));
}
else {
crc32c.update(d);
}
})
.on('end', () => resolve())
.on('error', reject);
});
return crc32c;
}
/**
* Generates a `CRC32C` from 4-byte base64-encoded data (string).
*
* @param value 4-byte base64-encoded data (string)
*/
static fromString(value) {
const buffer = Buffer.from(value, 'base64');
if (buffer.byteLength !== 4) {
throw new RangeError(CRC32C_EXCEPTION_MESSAGES.INVALID_INIT_BASE64_RANGE(buffer.byteLength));
}
return this.fromBuffer(buffer);
}
/**
* Generates a `CRC32C` from a safe, unsigned 32-bit integer.
*
* @param value an unsigned 32-bit integer
*/
static fromNumber(value) {
if (!Number.isSafeInteger(value) || value > 2 ** 32 || value < -(2 ** 32)) {
throw new RangeError(CRC32C_EXCEPTION_MESSAGES.INVALID_INIT_INTEGER(value));
}
return new CRC32C(value);
}
/**
* Generates a `CRC32C` from a variety of compatable types.
* Note: strings are treated as input, not as file paths to read from.
*
* @param value A number, 4-byte `ArrayBufferView`/`Buffer`/`TypedArray`, or 4-byte base64-encoded data (string)
*/
static from(value) {
if (typeof value === 'number') {
return this.fromNumber(value);
}
else if (typeof value === 'string') {
return this.fromString(value);
}
else if ('byteLength' in value) {
// `ArrayBuffer` | `Buffer` | `ArrayBufferView`
return this.fromBuffer(value);
}
else {
// `CRC32CValidator`/`CRC32C`-like
return this.fromString(value.toString());
}
}
}
_CRC32C_crc32c = new WeakMap();
CRC32C.CRC32C_EXTENSIONS = CRC32C_EXTENSIONS;
CRC32C.CRC32C_EXTENSION_TABLE = CRC32C_EXTENSION_TABLE;
export { CRC32C, CRC32C_DEFAULT_VALIDATOR_GENERATOR, CRC32C_EXCEPTION_MESSAGES, CRC32C_EXTENSIONS, CRC32C_EXTENSION_TABLE, };

View File

@@ -0,0 +1,982 @@
import { BodyResponseCallback, DecorateRequestOptions, GetConfig, MetadataCallback, ServiceObject, SetMetadataResponse } from './nodejs-common/index.js';
import * as resumableUpload from './resumable-upload.js';
import { Writable, Readable, PipelineSource } from 'stream';
import * as http from 'http';
import { PreconditionOptions, Storage } from './storage.js';
import { AvailableServiceObjectMethods, Bucket } from './bucket.js';
import { Acl, AclMetadata } from './acl.js';
import { GetSignedUrlResponse, GetSignedUrlCallback, URLSigner, SignerGetSignedUrlConfig, Query } from './signer.js';
import { Duplexify, GCCL_GCS_CMD_KEY } from './nodejs-common/util.js';
import { CRC32C, CRC32CValidatorGenerator } from './crc32c.js';
import { URL } from 'url';
import { BaseMetadata, DeleteCallback, DeleteOptions, GetResponse, InstanceResponseCallback, RequestResponse, SetMetadataOptions } from './nodejs-common/service-object.js';
import * as r from 'teeny-request';
export type GetExpirationDateResponse = [Date];
export interface GetExpirationDateCallback {
(err: Error | null, expirationDate?: Date | null, apiResponse?: unknown): void;
}
export interface PolicyDocument {
string: string;
base64: string;
signature: string;
}
export type SaveData = string | Buffer | Uint8Array | PipelineSource<string | Buffer | Uint8Array>;
export type GenerateSignedPostPolicyV2Response = [PolicyDocument];
export interface GenerateSignedPostPolicyV2Callback {
(err: Error | null, policy?: PolicyDocument): void;
}
export interface GenerateSignedPostPolicyV2Options {
equals?: string[] | string[][];
expires: string | number | Date;
startsWith?: string[] | string[][];
acl?: string;
successRedirect?: string;
successStatus?: string;
contentLengthRange?: {
min?: number;
max?: number;
};
/**
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string;
}
export interface PolicyFields {
[key: string]: string;
}
export interface GenerateSignedPostPolicyV4Options {
expires: string | number | Date;
bucketBoundHostname?: string;
virtualHostedStyle?: boolean;
conditions?: object[];
fields?: PolicyFields;
/**
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string;
}
export interface GenerateSignedPostPolicyV4Callback {
(err: Error | null, output?: SignedPostPolicyV4Output): void;
}
export type GenerateSignedPostPolicyV4Response = [SignedPostPolicyV4Output];
export interface SignedPostPolicyV4Output {
url: string;
fields: PolicyFields;
}
export interface GetSignedUrlConfig extends Pick<SignerGetSignedUrlConfig, 'host' | 'signingEndpoint'> {
action: 'read' | 'write' | 'delete' | 'resumable';
version?: 'v2' | 'v4';
virtualHostedStyle?: boolean;
cname?: string;
contentMd5?: string;
contentType?: string;
expires: string | number | Date;
accessibleAt?: string | number | Date;
extensionHeaders?: http.OutgoingHttpHeaders;
promptSaveAs?: string;
responseDisposition?: string;
responseType?: string;
queryParams?: Query;
}
export interface GetFileMetadataOptions {
userProject?: string;
}
export type GetFileMetadataResponse = [FileMetadata, unknown];
export interface GetFileMetadataCallback {
(err: Error | null, metadata?: FileMetadata, apiResponse?: unknown): void;
}
export interface GetFileOptions extends GetConfig {
userProject?: string;
generation?: number;
restoreToken?: string;
softDeleted?: boolean;
}
export type GetFileResponse = [File, unknown];
export interface GetFileCallback {
(err: Error | null, file?: File, apiResponse?: unknown): void;
}
export interface FileExistsOptions {
userProject?: string;
}
export type FileExistsResponse = [boolean];
export interface FileExistsCallback {
(err: Error | null, exists?: boolean): void;
}
export interface DeleteFileOptions {
ignoreNotFound?: boolean;
userProject?: string;
}
export type DeleteFileResponse = [unknown];
export interface DeleteFileCallback {
(err: Error | null, apiResponse?: unknown): void;
}
export type PredefinedAcl = 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
type PublicResumableUploadOptions = 'chunkSize' | 'highWaterMark' | 'isPartialUpload' | 'metadata' | 'origin' | 'offset' | 'predefinedAcl' | 'private' | 'public' | 'uri' | 'userProject';
export interface CreateResumableUploadOptions extends Pick<resumableUpload.UploadConfig, PublicResumableUploadOptions> {
/**
* A CRC32C to resume from when continuing a previous upload. It is recommended
* to capture the `crc32c` event from previous upload sessions to provide in
* subsequent requests in order to accurately track the upload. This is **required**
* when validating a final portion of the uploaded object.
*
* @see {@link CRC32C.from} for possible values.
*/
resumeCRC32C?: Parameters<(typeof CRC32C)['from']>[0];
preconditionOpts?: PreconditionOptions;
[GCCL_GCS_CMD_KEY]?: resumableUpload.UploadConfig[typeof GCCL_GCS_CMD_KEY];
}
export type CreateResumableUploadResponse = [string];
export interface CreateResumableUploadCallback {
(err: Error | null, uri?: string): void;
}
export interface CreateWriteStreamOptions extends CreateResumableUploadOptions {
contentType?: string;
gzip?: string | boolean;
resumable?: boolean;
timeout?: number;
validation?: string | boolean;
}
export interface MakeFilePrivateOptions {
metadata?: FileMetadata;
strict?: boolean;
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type MakeFilePrivateResponse = [unknown];
export type MakeFilePrivateCallback = SetFileMetadataCallback;
export interface IsPublicCallback {
(err: Error | null, resp?: boolean): void;
}
export type IsPublicResponse = [boolean];
export type MakeFilePublicResponse = [unknown];
export interface MakeFilePublicCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type MoveResponse = [unknown];
export interface MoveCallback {
(err: Error | null, destinationFile?: File | null, apiResponse?: unknown): void;
}
export interface MoveOptions {
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type MoveFileAtomicOptions = MoveOptions;
export type MoveFileAtomicCallback = MoveCallback;
export type MoveFileAtomicResponse = MoveResponse;
export type RenameOptions = MoveOptions;
export type RenameResponse = MoveResponse;
export type RenameCallback = MoveCallback;
export type RotateEncryptionKeyOptions = string | Buffer | EncryptionKeyOptions;
export interface EncryptionKeyOptions {
encryptionKey?: string | Buffer;
kmsKeyName?: string;
preconditionOpts?: PreconditionOptions;
}
export type RotateEncryptionKeyCallback = CopyCallback;
export type RotateEncryptionKeyResponse = CopyResponse;
export declare enum ActionToHTTPMethod {
read = "GET",
write = "PUT",
delete = "DELETE",
resumable = "POST"
}
/**
* @deprecated - no longer used
*/
export declare const STORAGE_POST_POLICY_BASE_URL = "https://storage.googleapis.com";
export interface FileOptions {
crc32cGenerator?: CRC32CValidatorGenerator;
encryptionKey?: string | Buffer;
generation?: number | string;
restoreToken?: string;
kmsKeyName?: string;
preconditionOpts?: PreconditionOptions;
userProject?: string;
}
export interface CopyOptions {
cacheControl?: string;
contentEncoding?: string;
contentType?: string;
contentDisposition?: string;
destinationKmsKeyName?: string;
metadata?: {
[key: string]: string | boolean | number | null;
};
predefinedAcl?: string;
token?: string;
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export type CopyResponse = [File, unknown];
export interface CopyCallback {
(err: Error | null, file?: File | null, apiResponse?: unknown): void;
}
export type DownloadResponse = [Buffer];
export type DownloadCallback = (err: RequestError | null, contents: Buffer) => void;
export interface DownloadOptions extends CreateReadStreamOptions {
destination?: string;
encryptionKey?: string | Buffer;
}
export interface CreateReadStreamOptions {
userProject?: string;
validation?: 'md5' | 'crc32c' | false | true;
start?: number;
end?: number;
decompress?: boolean;
[GCCL_GCS_CMD_KEY]?: string;
}
export interface SaveOptions extends CreateWriteStreamOptions {
onUploadProgress?: (progressEvent: any) => void;
}
export interface SaveCallback {
(err?: Error | null): void;
}
export interface SetFileMetadataOptions {
userProject?: string;
}
export interface SetFileMetadataCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export type SetFileMetadataResponse = [unknown];
export type SetStorageClassResponse = [unknown];
export interface SetStorageClassOptions {
userProject?: string;
preconditionOpts?: PreconditionOptions;
}
export interface SetStorageClassCallback {
(err?: Error | null, apiResponse?: unknown): void;
}
export interface RestoreOptions extends PreconditionOptions {
generation: number;
restoreToken?: string;
projection?: 'full' | 'noAcl';
}
export interface FileMetadata extends BaseMetadata {
acl?: AclMetadata[] | null;
bucket?: string;
cacheControl?: string;
componentCount?: number;
contentDisposition?: string;
contentEncoding?: string;
contentLanguage?: string;
contentType?: string;
crc32c?: string;
customerEncryption?: {
encryptionAlgorithm?: string;
keySha256?: string;
};
customTime?: string;
eventBasedHold?: boolean | null;
readonly eventBasedHoldReleaseTime?: string;
generation?: string | number;
restoreToken?: string;
hardDeleteTime?: string;
kmsKeyName?: string;
md5Hash?: string;
mediaLink?: string;
metadata?: {
[key: string]: string | boolean | number | null;
};
metageneration?: string | number;
name?: string;
owner?: {
entity?: string;
entityId?: string;
};
retention?: {
retainUntilTime?: string;
mode?: string;
} | null;
retentionExpirationTime?: string;
size?: string | number;
softDeleteTime?: string;
storageClass?: string;
temporaryHold?: boolean | null;
timeCreated?: string;
timeDeleted?: string;
timeStorageClassUpdated?: string;
updated?: string;
}
export declare class RequestError extends Error {
code?: string;
errors?: Error[];
}
export declare enum FileExceptionMessages {
EXPIRATION_TIME_NA = "An expiration time is not available.",
DESTINATION_NO_NAME = "Destination file should have a name.",
INVALID_VALIDATION_FILE_RANGE = "Cannot use validation with file ranges (start/end).",
MD5_NOT_AVAILABLE = "MD5 verification was specified, but is not available for the requested object. MD5 is not available for composite objects.",
EQUALS_CONDITION_TWO_ELEMENTS = "Equals condition must be an array of 2 elements.",
STARTS_WITH_TWO_ELEMENTS = "StartsWith condition must be an array of 2 elements.",
CONTENT_LENGTH_RANGE_MIN_MAX = "ContentLengthRange must have numeric min & max fields.",
DOWNLOAD_MISMATCH = "The downloaded data did not match the data from the server. To be sure the content is the same, you should download the file again.",
UPLOAD_MISMATCH_DELETE_FAIL = "The uploaded data did not match the data from the server.\n As a precaution, we attempted to delete the file, but it was not successful.\n To be sure the content is the same, you should try removing the file manually,\n then uploading the file again.\n \n\nThe delete attempt failed with this message:\n\n ",
UPLOAD_MISMATCH = "The uploaded data did not match the data from the server.\n As a precaution, the file has been deleted.\n To be sure the content is the same, you should try uploading the file again.",
MD5_RESUMED_UPLOAD = "MD5 cannot be used with a continued resumable upload as MD5 cannot be extended from an existing value",
MISSING_RESUME_CRC32C_FINAL_UPLOAD = "The CRC32C is missing for the final portion of a resumed upload, which is required for validation. Please provide `resumeCRC32C` if validation is required, or disable `validation`."
}
/**
* A File object is created from your {@link Bucket} object using
* {@link Bucket#file}.
*
* @class
*/
declare class File extends ServiceObject<File, FileMetadata> {
#private;
acl: Acl;
crc32cGenerator: CRC32CValidatorGenerator;
bucket: Bucket;
storage: Storage;
kmsKeyName?: string;
userProject?: string;
signer?: URLSigner;
name: string;
generation?: number;
restoreToken?: string;
parent: Bucket;
private encryptionKey?;
private encryptionKeyBase64?;
private encryptionKeyHash?;
private encryptionKeyInterceptor?;
private instanceRetryValue?;
instancePreconditionOpts?: PreconditionOptions;
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* An ACL consists of one or more entries, where each entry grants permissions
* to an entity. Permissions define the actions that can be performed against
* an object or bucket (for example, `READ` or `WRITE`); the entity defines
* who the permission applies to (for example, a specific user or group of
* users).
*
* The `acl` object on a File instance provides methods to get you a list of
* the ACLs defined on your bucket, as well as set, update, and delete them.
*
* See {@link http://goo.gl/6qBBPO| About Access Control lists}
*
* @name File#acl
* @mixes Acl
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
* //-
* // Make a file publicly readable.
* //-
* const options = {
* entity: 'allUsers',
* role: storage.acl.READER_ROLE
* };
*
* file.acl.add(options, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* file.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
/**
* The API-formatted resource description of the file.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name File#metadata
* @type {object}
*/
/**
* The file's name.
* @name File#name
* @type {string}
*/
/**
* @callback Crc32cGeneratorToStringCallback
* A method returning the CRC32C as a base64-encoded string.
*
* @returns {string}
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
/**
* @callback Crc32cGeneratorValidateCallback
* A method validating a base64-encoded CRC32C string.
*
* @param {string} [value] base64-encoded CRC32C string to validate
* @returns {boolean}
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
**/
/**
* @callback Crc32cGeneratorUpdateCallback
* A method for passing `Buffer`s for CRC32C generation.
*
* @param {Buffer} [data] data to update CRC32C value with
* @returns {undefined}
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
**/
/**
* @typedef {object} CRC32CValidator
* @property {Crc32cGeneratorToStringCallback}
* @property {Crc32cGeneratorValidateCallback}
* @property {Crc32cGeneratorUpdateCallback}
*/
/**
* @callback Crc32cGeneratorCallback
* @returns {CRC32CValidator}
*/
/**
* @typedef {object} FileOptions Options passed to the File constructor.
* @property {string} [encryptionKey] A custom encryption key.
* @property {number} [generation] Generation to scope the file to.
* @property {string} [kmsKeyName] Cloud KMS Key used to encrypt this
* object, if the object is encrypted by such a key. Limited availability;
* usable only by enabled projects.
* @property {string} [userProject] The ID of the project which will be
* billed for all requests made from File object.
* @property {Crc32cGeneratorCallback} [callback] A function that generates a CRC32C Validator. Defaults to {@link CRC32C}
*/
/**
* Constructs a file object.
*
* @param {Bucket} bucket The Bucket instance this file is
* attached to.
* @param {string} name The name of the remote file.
* @param {FileOptions} [options] Configuration options.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
* ```
*/
constructor(bucket: Bucket, name: string, options?: FileOptions);
/**
* The object's Cloud Storage URI (`gs://`)
*
* @example
* ```ts
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const file = bucket.file('image.png');
*
* // `gs://my-bucket/image.png`
* const href = file.cloudStorageURI.href;
* ```
*/
get cloudStorageURI(): URL;
/**
* A helper method for determining if a request should be retried based on preconditions.
* This should only be used for methods where the idempotency is determined by
* `ifGenerationMatch`
* @private
*
* A request should not be retried under the following conditions:
* - if precondition option `ifGenerationMatch` is not set OR
* - if `idempotencyStrategy` is set to `RetryNever`
*/
private shouldRetryBasedOnPreconditionAndIdempotencyStrat;
copy(destination: string | Bucket | File, options?: CopyOptions): Promise<CopyResponse>;
copy(destination: string | Bucket | File, callback: CopyCallback): void;
copy(destination: string | Bucket | File, options: CopyOptions, callback: CopyCallback): void;
/**
* @typedef {object} CreateReadStreamOptions Configuration options for File#createReadStream.
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
* @property {string|boolean} [validation] Possible values: `"md5"`,
* `"crc32c"`, or `false`. By default, data integrity is validated with a
* CRC32c checksum. You may use MD5 if preferred, but that hash is not
* supported for composite objects. An error will be raised if MD5 is
* specified but is not available. You may also choose to skip validation
* completely, however this is **not recommended**.
* @property {number} [start] A byte offset to begin the file's download
* from. Default is 0. NOTE: Byte ranges are inclusive; that is,
* `options.start = 0` and `options.end = 999` represent the first 1000
* bytes in a file or object. NOTE: when specifying a byte range, data
* integrity is not available.
* @property {number} [end] A byte offset to stop reading the file at.
* NOTE: Byte ranges are inclusive; that is, `options.start = 0` and
* `options.end = 999` represent the first 1000 bytes in a file or object.
* NOTE: when specifying a byte range, data integrity is not available.
* @property {boolean} [decompress=true] Disable auto decompression of the
* received data. By default this option is set to `true`.
* Applicable in cases where the data was uploaded with
* `gzip: true` option. See {@link File#createWriteStream}.
*/
/**
* Create a readable stream to read the contents of the remote file. It can be
* piped to a writable stream or listened to for 'data' events to read a
* file's contents.
*
* In the unlikely event there is a mismatch between what you downloaded and
* the version in your Bucket, your error handler will receive an error with
* code "CONTENT_DOWNLOAD_MISMATCH". If you receive this error, the best
* recourse is to try downloading the file again.
*
* NOTE: Readable streams will emit the `end` event when the file is fully
* downloaded.
*
* @param {CreateReadStreamOptions} [options] Configuration options.
* @returns {ReadableStream}
*
* @example
* ```
* //-
* // <h4>Downloading a File</h4>
* //
* // The example below demonstrates how we can reference a remote file, then
* // pipe its contents to a local file. This is effectively creating a local
* // backup of your remote data.
* //-
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* const fs = require('fs');
* const remoteFile = bucket.file('image.png');
* const localFilename = '/Users/stephen/Photos/image.png';
*
* remoteFile.createReadStream()
* .on('error', function(err) {})
* .on('response', function(response) {
* // Server connected and responded with the specified status and headers.
* })
* .on('end', function() {
* // The file is fully downloaded.
* })
* .pipe(fs.createWriteStream(localFilename));
*
* //-
* // To limit the downloaded data to only a byte range, pass an options
* // object.
* //-
* const logFile = myBucket.file('access_log');
* logFile.createReadStream({
* start: 10000,
* end: 20000
* })
* .on('error', function(err) {})
* .pipe(fs.createWriteStream('/Users/stephen/logfile.txt'));
*
* //-
* // To read a tail byte range, specify only `options.end` as a negative
* // number.
* //-
* const logFile = myBucket.file('access_log');
* logFile.createReadStream({
* end: -100
* })
* .on('error', function(err) {})
* .pipe(fs.createWriteStream('/Users/stephen/logfile.txt'));
* ```
*/
createReadStream(options?: CreateReadStreamOptions): Readable;
createResumableUpload(options?: CreateResumableUploadOptions): Promise<CreateResumableUploadResponse>;
createResumableUpload(options: CreateResumableUploadOptions, callback: CreateResumableUploadCallback): void;
createResumableUpload(callback: CreateResumableUploadCallback): void;
/**
* @typedef {object} CreateWriteStreamOptions Configuration options for File#createWriteStream().
* @property {string} [contentType] Alias for
* `options.metadata.contentType`. If set to `auto`, the file name is used
* to determine the contentType.
* @property {string|boolean} [gzip] If true, automatically gzip the file.
* If set to `auto`, the contentType is used to determine if the file
* should be gzipped. This will set `options.metadata.contentEncoding` to
* `gzip` if necessary.
* @property {object} [metadata] See the examples below or
* {@link https://cloud.google.com/storage/docs/json_api/v1/objects/insert#request_properties_JSON| Objects: insert request body}
* for more details.
* @property {number} [offset] The starting byte of the upload stream, for
* resuming an interrupted upload. Defaults to 0.
* @property {string} [predefinedAcl] Apply a predefined set of access
* controls to this object.
*
* Acceptable values are:
* - **`authenticatedRead`** - Object owner gets `OWNER` access, and
* `allAuthenticatedUsers` get `READER` access.
*
* - **`bucketOwnerFullControl`** - Object owner gets `OWNER` access, and
* project team owners get `OWNER` access.
*
* - **`bucketOwnerRead`** - Object owner gets `OWNER` access, and project
* team owners get `READER` access.
*
* - **`private`** - Object owner gets `OWNER` access.
*
* - **`projectPrivate`** - Object owner gets `OWNER` access, and project
* team members get access according to their roles.
*
* - **`publicRead`** - Object owner gets `OWNER` access, and `allUsers`
* get `READER` access.
* @property {boolean} [private] Make the uploaded file private. (Alias for
* `options.predefinedAcl = 'private'`)
* @property {boolean} [public] Make the uploaded file public. (Alias for
* `options.predefinedAcl = 'publicRead'`)
* @property {boolean} [resumable] Force a resumable upload. NOTE: When
* working with streams, the file format and size is unknown until it's
* completely consumed. Because of this, it's best for you to be explicit
* for what makes sense given your input.
* @property {number} [timeout=60000] Set the HTTP request timeout in
* milliseconds. This option is not available for resumable uploads.
* Default: `60000`
* @property {string} [uri] The URI for an already-created resumable
* upload. See {@link File#createResumableUpload}.
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
* @property {string|boolean} [validation] Possible values: `"md5"`,
* `"crc32c"`, or `false`. By default, data integrity is validated with a
* CRC32c checksum. You may use MD5 if preferred, but that hash is not
* supported for composite objects. An error will be raised if MD5 is
* specified but is not available. You may also choose to skip validation
* completely, however this is **not recommended**. In addition to specifying
* validation type, providing `metadata.crc32c` or `metadata.md5Hash` will
* cause the server to perform validation in addition to client validation.
* NOTE: Validation is automatically skipped for objects that were
* uploaded using the `gzip` option and have already compressed content.
*/
/**
* Create a writable stream to overwrite the contents of the file in your
* bucket.
*
* A File object can also be used to create files for the first time.
*
* Resumable uploads are automatically enabled and must be shut off explicitly
* by setting `options.resumable` to `false`.
*
*
* <p class="notice">
* There is some overhead when using a resumable upload that can cause
* noticeable performance degradation while uploading a series of small
* files. When uploading files less than 10MB, it is recommended that the
* resumable feature is disabled.
* </p>
*
* NOTE: Writable streams will emit the `finish` event when the file is fully
* uploaded.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload Upload Options (Simple or Resumable)}
* See {@link https://cloud.google.com/storage/docs/json_api/v1/objects/insert Objects: insert API Documentation}
*
* @param {CreateWriteStreamOptions} [options] Configuration options.
* @returns {WritableStream}
*
* @example
* ```
* const fs = require('fs');
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const file = myBucket.file('my-file');
*
* //-
* // <h4>Uploading a File</h4>
* //
* // Now, consider a case where we want to upload a file to your bucket. You
* // have the option of using {@link Bucket#upload}, but that is just
* // a convenience method which will do the following.
* //-
* fs.createReadStream('/Users/stephen/Photos/birthday-at-the-zoo/panda.jpg')
* .pipe(file.createWriteStream())
* .on('error', function(err) {})
* .on('finish', function() {
* // The file upload is complete.
* });
*
* //-
* // <h4>Uploading a File with gzip compression</h4>
* //-
* fs.createReadStream('/Users/stephen/site/index.html')
* .pipe(file.createWriteStream({ gzip: true }))
* .on('error', function(err) {})
* .on('finish', function() {
* // The file upload is complete.
* });
*
* //-
* // Downloading the file with `createReadStream` will automatically decode
* // the file.
* //-
*
* //-
* // <h4>Uploading a File with Metadata</h4>
* //
* // One last case you may run into is when you want to upload a file to your
* // bucket and set its metadata at the same time. Like above, you can use
* // {@link Bucket#upload} to do this, which is just a wrapper around
* // the following.
* //-
* fs.createReadStream('/Users/stephen/Photos/birthday-at-the-zoo/panda.jpg')
* .pipe(file.createWriteStream({
* metadata: {
* contentType: 'image/jpeg',
* metadata: {
* custom: 'metadata'
* }
* }
* }))
* .on('error', function(err) {})
* .on('finish', function() {
* // The file upload is complete.
* });
* ```
*
* //-
* // <h4>Continuing a Resumable Upload</h4>
* //
* // One can capture a `uri` from a resumable upload to reuse later.
* // Additionally, for validation, one can also capture and pass `crc32c`.
* //-
* let uri: string | undefined = undefined;
* let resumeCRC32C: string | undefined = undefined;
*
* fs.createWriteStream()
* .on('uri', link => {uri = link})
* .on('crc32', crc32c => {resumeCRC32C = crc32c});
*
* // later...
* fs.createWriteStream({uri, resumeCRC32C});
*/
createWriteStream(options?: CreateWriteStreamOptions): Writable;
/**
* Delete the object.
*
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
delete(options?: DeleteOptions): Promise<[r.Response]>;
delete(options: DeleteOptions, callback: DeleteCallback): void;
delete(callback: DeleteCallback): void;
download(options?: DownloadOptions): Promise<DownloadResponse>;
download(options: DownloadOptions, callback: DownloadCallback): void;
download(callback: DownloadCallback): void;
/**
* The Storage API allows you to use a custom key for server-side encryption.
*
* See {@link https://cloud.google.com/storage/docs/encryption#customer-supplied| Customer-supplied Encryption Keys}
*
* @param {string|buffer} encryptionKey An AES-256 encryption key.
* @returns {File}
*
* @example
* ```
* const crypto = require('crypto');
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const encryptionKey = crypto.randomBytes(32);
*
* const fileWithCustomEncryption = myBucket.file('my-file');
* fileWithCustomEncryption.setEncryptionKey(encryptionKey);
*
* const fileWithoutCustomEncryption = myBucket.file('my-file');
*
* fileWithCustomEncryption.save('data', function(err) {
* // Try to download with the File object that hasn't had
* // `setEncryptionKey()` called:
* fileWithoutCustomEncryption.download(function(err) {
* // We will receive an error:
* // err.message === 'Bad Request'
*
* // Try again with the File object we called `setEncryptionKey()` on:
* fileWithCustomEncryption.download(function(err, contents) {
* // contents.toString() === 'data'
* });
* });
* });
*
* ```
* @example <caption>include:samples/encryption.js</caption>
* region_tag:storage_upload_encrypted_file
* Example of uploading an encrypted file:
*
* @example <caption>include:samples/encryption.js</caption>
* region_tag:storage_download_encrypted_file
* Example of downloading an encrypted file:
*/
setEncryptionKey(encryptionKey: string | Buffer): this;
/**
* Gets a reference to a Cloud Storage {@link File} file from the provided URL in string format.
* @param {string} publicUrlOrGsUrl the URL as a string. Must be of the format gs://bucket/file
* or https://storage.googleapis.com/bucket/file.
* @param {Storage} storageInstance an instance of a Storage object.
* @param {FileOptions} [options] Configuration options
* @returns {File}
*/
static from(publicUrlOrGsUrl: string, storageInstance: Storage, options?: FileOptions): File;
get(options?: GetFileOptions): Promise<GetResponse<File>>;
get(callback: InstanceResponseCallback<File>): void;
get(options: GetFileOptions, callback: InstanceResponseCallback<File>): void;
getExpirationDate(): Promise<GetExpirationDateResponse>;
getExpirationDate(callback: GetExpirationDateCallback): void;
generateSignedPostPolicyV2(options: GenerateSignedPostPolicyV2Options): Promise<GenerateSignedPostPolicyV2Response>;
generateSignedPostPolicyV2(options: GenerateSignedPostPolicyV2Options, callback: GenerateSignedPostPolicyV2Callback): void;
generateSignedPostPolicyV2(callback: GenerateSignedPostPolicyV2Callback): void;
generateSignedPostPolicyV4(options: GenerateSignedPostPolicyV4Options): Promise<GenerateSignedPostPolicyV4Response>;
generateSignedPostPolicyV4(options: GenerateSignedPostPolicyV4Options, callback: GenerateSignedPostPolicyV4Callback): void;
generateSignedPostPolicyV4(callback: GenerateSignedPostPolicyV4Callback): void;
getSignedUrl(cfg: GetSignedUrlConfig): Promise<GetSignedUrlResponse>;
getSignedUrl(cfg: GetSignedUrlConfig, callback: GetSignedUrlCallback): void;
isPublic(): Promise<IsPublicResponse>;
isPublic(callback: IsPublicCallback): void;
makePrivate(options?: MakeFilePrivateOptions): Promise<MakeFilePrivateResponse>;
makePrivate(callback: MakeFilePrivateCallback): void;
makePrivate(options: MakeFilePrivateOptions, callback: MakeFilePrivateCallback): void;
makePublic(): Promise<MakeFilePublicResponse>;
makePublic(callback: MakeFilePublicCallback): void;
/**
* The public URL of this File
* Use {@link File#makePublic} to enable anonymous access via the returned URL.
*
* @returns {string}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* const file = bucket.file('my-file');
*
* // publicUrl will be "https://storage.googleapis.com/albums/my-file"
* const publicUrl = file.publicUrl();
* ```
*/
publicUrl(): string;
moveFileAtomic(destination: string | File, options?: MoveFileAtomicOptions): Promise<MoveFileAtomicResponse>;
moveFileAtomic(destination: string | File, callback: MoveFileAtomicCallback): void;
moveFileAtomic(destination: string | File, options: MoveFileAtomicOptions, callback: MoveFileAtomicCallback): void;
move(destination: string | Bucket | File, options?: MoveOptions): Promise<MoveResponse>;
move(destination: string | Bucket | File, callback: MoveCallback): void;
move(destination: string | Bucket | File, options: MoveOptions, callback: MoveCallback): void;
rename(destinationFile: string | File, options?: RenameOptions): Promise<RenameResponse>;
rename(destinationFile: string | File, callback: RenameCallback): void;
rename(destinationFile: string | File, options: RenameOptions, callback: RenameCallback): void;
/**
* @typedef {object} RestoreOptions Options for File#restore(). See an
* {@link https://cloud.google.com/storage/docs/json_api/v1/objects#resource| Object resource}.
* @param {string} [userProject] The ID of the project which will be
* billed for the request.
* @param {number} [generation] If present, selects a specific revision of this object.
* @param {string} [restoreToken] Returns an option that must be specified when getting a soft-deleted object from an HNS-enabled
* bucket that has a naming and generation conflict with another object in the same bucket.
* @param {string} [projection] Specifies the set of properties to return. If used, must be 'full' or 'noAcl'.
* @param {string | number} [ifGenerationMatch] Request proceeds if the generation of the target resource
* matches the value used in the precondition.
* If the values don't match, the request fails with a 412 Precondition Failed response.
* @param {string | number} [ifGenerationNotMatch] Request proceeds if the generation of the target resource does
* not match the value used in the precondition. If the values match, the request fails with a 304 Not Modified response.
* @param {string | number} [ifMetagenerationMatch] Request proceeds if the meta-generation of the target resource
* matches the value used in the precondition.
* If the values don't match, the request fails with a 412 Precondition Failed response.
* @param {string | number} [ifMetagenerationNotMatch] Request proceeds if the meta-generation of the target resource does
* not match the value used in the precondition. If the values match, the request fails with a 304 Not Modified response.
*/
/**
* Restores a soft-deleted file
* @param {RestoreOptions} options Restore options.
* @returns {Promise<File>}
*/
restore(options: RestoreOptions): Promise<File>;
request(reqOpts: DecorateRequestOptions): Promise<RequestResponse>;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
rotateEncryptionKey(options?: RotateEncryptionKeyOptions): Promise<RotateEncryptionKeyResponse>;
rotateEncryptionKey(callback: RotateEncryptionKeyCallback): void;
rotateEncryptionKey(options: RotateEncryptionKeyOptions, callback: RotateEncryptionKeyCallback): void;
save(data: SaveData, options?: SaveOptions): Promise<void>;
save(data: SaveData, callback: SaveCallback): void;
save(data: SaveData, options: SaveOptions, callback: SaveCallback): void;
setMetadata(metadata: FileMetadata, options?: SetMetadataOptions): Promise<SetMetadataResponse<FileMetadata>>;
setMetadata(metadata: FileMetadata, callback: MetadataCallback<FileMetadata>): void;
setMetadata(metadata: FileMetadata, options: SetMetadataOptions, callback: MetadataCallback<FileMetadata>): void;
setStorageClass(storageClass: string, options?: SetStorageClassOptions): Promise<SetStorageClassResponse>;
setStorageClass(storageClass: string, options: SetStorageClassOptions, callback: SetStorageClassCallback): void;
setStorageClass(storageClass: string, callback?: SetStorageClassCallback): void;
/**
* Set a user project to be billed for all requests made from this File
* object.
*
* @param {string} userProject The user project.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('albums');
* const file = bucket.file('my-file');
*
* file.setUserProject('grape-spaceship-123');
* ```
*/
setUserProject(userProject: string): void;
/**
* This creates a resumable-upload upload stream.
*
* @param {Duplexify} stream - Duplexify stream of data to pipe to the file.
* @param {object=} options - Configuration object.
*
* @private
*/
startResumableUpload_(dup: Duplexify, options?: CreateResumableUploadOptions): void;
/**
* Takes a readable stream and pipes it to a remote file. Unlike
* `startResumableUpload_`, which uses the resumable upload technique, this
* method uses a simple upload (all or nothing).
*
* @param {Duplexify} dup - Duplexify stream of data to pipe to the file.
* @param {object=} options - Configuration object.
*
* @private
*/
startSimpleUpload_(dup: Duplexify, options?: CreateWriteStreamOptions): void;
disableAutoRetryConditionallyIdempotent_(coreOpts: any, methodType: AvailableServiceObjectMethods, localPreconditionOptions?: PreconditionOptions): void;
private getBufferFromReadable;
}
/**
* Reference to the {@link File} class.
* @name module:@google-cloud/storage.File
* @see File
*/
export { File };

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,35 @@
import { Transform } from 'stream';
import { CRC32CValidatorGenerator, CRC32CValidator } from './crc32c.js';
interface HashStreamValidatorOptions {
/** Enables CRC32C calculation. To validate a provided value use `crc32cExpected`. */
crc32c: boolean;
/** Enables MD5 calculation. To validate a provided value use `md5Expected`. */
md5: boolean;
/** A CRC32C instance for validation. To validate a provided value use `crc32cExpected`. */
crc32cInstance: CRC32CValidator;
/** Set a custom CRC32C generator. Used if `crc32cInstance` has not been provided. */
crc32cGenerator: CRC32CValidatorGenerator;
/** Sets the expected CRC32C value to verify once all data has been consumed. Also sets the `crc32c` option to `true` */
crc32cExpected?: string;
/** Sets the expected MD5 value to verify once all data has been consumed. Also sets the `md5` option to `true` */
md5Expected?: string;
/** Indicates whether or not to run a validation check or only update the hash values */
updateHashesOnly?: boolean;
}
declare class HashStreamValidator extends Transform {
#private;
readonly crc32cEnabled: boolean;
readonly md5Enabled: boolean;
readonly crc32cExpected: string | undefined;
readonly md5Expected: string | undefined;
readonly updateHashesOnly: boolean;
constructor(options?: Partial<HashStreamValidatorOptions>);
/**
* Return the current CRC32C value, if available.
*/
get crc32c(): string | undefined;
_flush(callback: (error?: Error | null | undefined) => void): void;
_transform(chunk: Buffer, encoding: BufferEncoding, callback: (e?: Error) => void): void;
test(hash: 'crc32c' | 'md5', sum: Buffer | string): boolean;
}
export { HashStreamValidator, HashStreamValidatorOptions };

View File

@@ -0,0 +1,116 @@
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _HashStreamValidator_crc32cHash, _HashStreamValidator_md5Hash, _HashStreamValidator_md5Digest;
import { createHash } from 'crypto';
import { Transform } from 'stream';
import { CRC32C_DEFAULT_VALIDATOR_GENERATOR, } from './crc32c.js';
import { FileExceptionMessages, RequestError } from './file.js';
class HashStreamValidator extends Transform {
constructor(options = {}) {
super();
this.updateHashesOnly = false;
_HashStreamValidator_crc32cHash.set(this, undefined);
_HashStreamValidator_md5Hash.set(this, undefined);
_HashStreamValidator_md5Digest.set(this, '');
this.crc32cEnabled = !!options.crc32c;
this.md5Enabled = !!options.md5;
this.updateHashesOnly = !!options.updateHashesOnly;
this.crc32cExpected = options.crc32cExpected;
this.md5Expected = options.md5Expected;
if (this.crc32cEnabled) {
if (options.crc32cInstance) {
__classPrivateFieldSet(this, _HashStreamValidator_crc32cHash, options.crc32cInstance, "f");
}
else {
const crc32cGenerator = options.crc32cGenerator || CRC32C_DEFAULT_VALIDATOR_GENERATOR;
__classPrivateFieldSet(this, _HashStreamValidator_crc32cHash, crc32cGenerator(), "f");
}
}
if (this.md5Enabled) {
__classPrivateFieldSet(this, _HashStreamValidator_md5Hash, createHash('md5'), "f");
}
}
/**
* Return the current CRC32C value, if available.
*/
get crc32c() {
var _a;
return (_a = __classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f")) === null || _a === void 0 ? void 0 : _a.toString();
}
_flush(callback) {
if (__classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f")) {
__classPrivateFieldSet(this, _HashStreamValidator_md5Digest, __classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f").digest('base64'), "f");
}
if (this.updateHashesOnly) {
callback();
return;
}
// If we're doing validation, assume the worst-- a data integrity
// mismatch. If not, these tests won't be performed, and we can assume
// the best.
// We must check if the server decompressed the data on serve because hash
// validation is not possible in this case.
let failed = this.crc32cEnabled || this.md5Enabled;
if (this.crc32cEnabled && this.crc32cExpected) {
failed = !this.test('crc32c', this.crc32cExpected);
}
if (this.md5Enabled && this.md5Expected) {
failed = !this.test('md5', this.md5Expected);
}
if (failed) {
const mismatchError = new RequestError(FileExceptionMessages.DOWNLOAD_MISMATCH);
mismatchError.code = 'CONTENT_DOWNLOAD_MISMATCH';
callback(mismatchError);
}
else {
callback();
}
}
_transform(chunk, encoding, callback) {
this.push(chunk, encoding);
try {
if (__classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f"))
__classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f").update(chunk);
if (__classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f"))
__classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f").update(chunk);
callback();
}
catch (e) {
callback(e);
}
}
test(hash, sum) {
const check = Buffer.isBuffer(sum) ? sum.toString('base64') : sum;
if (hash === 'crc32c' && __classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f")) {
return __classPrivateFieldGet(this, _HashStreamValidator_crc32cHash, "f").validate(check);
}
if (hash === 'md5' && __classPrivateFieldGet(this, _HashStreamValidator_md5Hash, "f")) {
return __classPrivateFieldGet(this, _HashStreamValidator_md5Digest, "f") === check;
}
return false;
}
}
_HashStreamValidator_crc32cHash = new WeakMap(), _HashStreamValidator_md5Hash = new WeakMap(), _HashStreamValidator_md5Digest = new WeakMap();
export { HashStreamValidator };

View File

@@ -0,0 +1,93 @@
import { ServiceObject, MetadataCallback, SetMetadataResponse } from './nodejs-common/index.js';
import { BaseMetadata, SetMetadataOptions } from './nodejs-common/service-object.js';
import { Storage } from './storage.js';
export interface HmacKeyOptions {
projectId?: string;
}
export interface HmacKeyMetadata extends BaseMetadata {
accessId?: string;
etag?: string;
projectId?: string;
serviceAccountEmail?: string;
state?: string;
timeCreated?: string;
updated?: string;
}
export interface SetHmacKeyMetadataOptions {
/**
* This parameter is currently ignored.
*/
userProject?: string;
}
export interface SetHmacKeyMetadata {
state?: 'ACTIVE' | 'INACTIVE';
etag?: string;
}
export interface HmacKeyMetadataCallback {
(err: Error | null, metadata?: HmacKeyMetadata, apiResponse?: unknown): void;
}
export type HmacKeyMetadataResponse = [HmacKeyMetadata, unknown];
/**
* The API-formatted resource description of the HMAC key.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name HmacKey#metadata
* @type {object}
*/
/**
* An HmacKey object contains metadata of an HMAC key created from a
* service account through the {@link Storage} client using
* {@link Storage#createHmacKey}.
*
* See {@link https://cloud.google.com/storage/docs/authentication/hmackeys| HMAC keys documentation}
*
* @class
*/
export declare class HmacKey extends ServiceObject<HmacKey, HmacKeyMetadata> {
/**
* A reference to the {@link Storage} associated with this {@link HmacKey}
* instance.
* @name HmacKey#storage
* @type {Storage}
*/
storage: Storage;
private instanceRetryValue?;
/**
* @typedef {object} HmacKeyOptions
* @property {string} [projectId] The project ID of the project that owns
* the service account of the requested HMAC key. If not provided,
* the project ID used to instantiate the Storage client will be used.
*/
/**
* Constructs an HmacKey object.
*
* Note: this only create a local reference to an HMAC key, to create
* an HMAC key, use {@link Storage#createHmacKey}.
*
* @param {Storage} storage The Storage instance this HMAC key is
* attached to.
* @param {string} accessId The unique accessId for this HMAC key.
* @param {HmacKeyOptions} options Constructor configurations.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const hmacKey = storage.hmacKey('access-id');
* ```
*/
constructor(storage: Storage, accessId: string, options?: HmacKeyOptions);
/**
* Set the metadata for this object.
*
* @param {object} metadata - The metadata to set on this object.
* @param {object=} options - Configuration options.
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
setMetadata(metadata: HmacKeyMetadata, options?: SetMetadataOptions): Promise<SetMetadataResponse<HmacKeyMetadata>>;
setMetadata(metadata: HmacKeyMetadata, callback: MetadataCallback<HmacKeyMetadata>): void;
setMetadata(metadata: HmacKeyMetadata, options: SetMetadataOptions, callback: MetadataCallback<HmacKeyMetadata>): void;
}

View File

@@ -0,0 +1,332 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ServiceObject, } from './nodejs-common/index.js';
import { IdempotencyStrategy } from './storage.js';
import { promisifyAll } from '@google-cloud/promisify';
/**
* The API-formatted resource description of the HMAC key.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name HmacKey#metadata
* @type {object}
*/
/**
* An HmacKey object contains metadata of an HMAC key created from a
* service account through the {@link Storage} client using
* {@link Storage#createHmacKey}.
*
* See {@link https://cloud.google.com/storage/docs/authentication/hmackeys| HMAC keys documentation}
*
* @class
*/
export class HmacKey extends ServiceObject {
/**
* @typedef {object} HmacKeyOptions
* @property {string} [projectId] The project ID of the project that owns
* the service account of the requested HMAC key. If not provided,
* the project ID used to instantiate the Storage client will be used.
*/
/**
* Constructs an HmacKey object.
*
* Note: this only create a local reference to an HMAC key, to create
* an HMAC key, use {@link Storage#createHmacKey}.
*
* @param {Storage} storage The Storage instance this HMAC key is
* attached to.
* @param {string} accessId The unique accessId for this HMAC key.
* @param {HmacKeyOptions} options Constructor configurations.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const hmacKey = storage.hmacKey('access-id');
* ```
*/
constructor(storage, accessId, options) {
const methods = {
/**
* @typedef {object} DeleteHmacKeyOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* @typedef {array} DeleteHmacKeyResponse
* @property {object} 0 The full API response.
*/
/**
* @callback DeleteHmacKeyCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
/**
* Deletes an HMAC key.
* Key state must be set to `INACTIVE` prior to deletion.
* Caution: HMAC keys cannot be recovered once you delete them.
*
* The authenticated user must have `storage.hmacKeys.delete` permission for the project in which the key exists.
*
* @method HmacKey#delete
* @param {DeleteHmacKeyOptions} [options] Configuration options.
* @param {DeleteHmacKeyCallback} [callback] Callback function.
* @returns {Promise<DeleteHmacKeyResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Delete HMAC key after making the key inactive.
* //-
* const hmacKey = storage.hmacKey('ACCESS_ID');
* hmacKey.setMetadata({state: 'INACTIVE'}, (err, hmacKeyMetadata) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* hmacKey.delete((err) => {
* if (err) {
* console.error(err);
* return;
* }
* // The HMAC key is deleted.
* });
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* const hmacKey = storage.hmacKey('ACCESS_ID');
* hmacKey
* .setMetadata({state: 'INACTIVE'})
* .then(() => {
* return hmacKey.delete();
* });
* ```
*/
delete: true,
/**
* @callback GetHmacKeyCallback
* @param {?Error} err Request error, if any.
* @param {HmacKey} hmacKey this {@link HmacKey} instance.
* @param {object} apiResponse The full API response.
*/
/**
* @typedef {array} GetHmacKeyResponse
* @property {HmacKey} 0 This {@link HmacKey} instance.
* @property {object} 1 The full API response.
*/
/**
* @typedef {object} GetHmacKeyOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* Retrieves and populate an HMAC key's metadata, and return
* this {@link HmacKey} instance.
*
* HmacKey.get() does not give the HMAC key secret, as
* it is only returned on creation.
*
* The authenticated user must have `storage.hmacKeys.get` permission
* for the project in which the key exists.
*
* @method HmacKey#get
* @param {GetHmacKeyOptions} [options] Configuration options.
* @param {GetHmacKeyCallback} [callback] Callback function.
* @returns {Promise<GetHmacKeyResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Get the HmacKey's Metadata.
* //-
* storage.hmacKey('ACCESS_ID')
* .get((err, hmacKey) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* // do something with the returned HmacKey object.
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* storage.hmacKey('ACCESS_ID')
* .get()
* .then((data) => {
* const hmacKey = data[0];
* });
* ```
*/
get: true,
/**
* @typedef {object} GetHmacKeyMetadataOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* Retrieves and populate an HMAC key's metadata, and return
* the HMAC key's metadata as an object.
*
* HmacKey.getMetadata() does not give the HMAC key secret, as
* it is only returned on creation.
*
* The authenticated user must have `storage.hmacKeys.get` permission
* for the project in which the key exists.
*
* @method HmacKey#getMetadata
* @param {GetHmacKeyMetadataOptions} [options] Configuration options.
* @param {HmacKeyMetadataCallback} [callback] Callback function.
* @returns {Promise<HmacKeyMetadataResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* //-
* // Get the HmacKey's metadata and populate to the metadata property.
* //-
* storage.hmacKey('ACCESS_ID')
* .getMetadata((err, hmacKeyMetadata) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* console.log(hmacKeyMetadata);
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* storage.hmacKey('ACCESS_ID')
* .getMetadata()
* .then((data) => {
* const hmacKeyMetadata = data[0];
* console.log(hmacKeyMetadata);
* });
* ```
*/
getMetadata: true,
/**
* @typedef {object} SetHmacKeyMetadata Subset of {@link HmacKeyMetadata} to update.
* @property {string} state New state of the HmacKey. Either 'ACTIVE' or 'INACTIVE'.
* @property {string} [etag] Include an etag from a previous get HMAC key request
* to perform safe read-modify-write.
*/
/**
* @typedef {object} SetHmacKeyMetadataOptions
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* @callback HmacKeyMetadataCallback
* @param {?Error} err Request error, if any.
* @param {HmacKeyMetadata} metadata The updated {@link HmacKeyMetadata} object.
* @param {object} apiResponse The full API response.
*/
/**
* @typedef {array} HmacKeyMetadataResponse
* @property {HmacKeyMetadata} 0 The updated {@link HmacKeyMetadata} object.
* @property {object} 1 The full API response.
*/
/**
* Updates the state of an HMAC key. See {@link SetHmacKeyMetadata} for
* valid states.
*
* @method HmacKey#setMetadata
* @param {SetHmacKeyMetadata} metadata The new metadata.
* @param {SetHmacKeyMetadataOptions} [options] Configuration options.
* @param {HmacKeyMetadataCallback} [callback] Callback function.
* @returns {Promise<HmacKeyMetadataResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
*
* const metadata = {
* state: 'INACTIVE',
* };
*
* storage.hmacKey('ACCESS_ID')
* .setMetadata(metadata, (err, hmacKeyMetadata) => {
* if (err) {
* // The request was an error.
* console.error(err);
* return;
* }
* console.log(hmacKeyMetadata);
* });
*
* //-
* // If the callback is omitted, a promise is returned.
* //-
* storage.hmacKey('ACCESS_ID')
* .setMetadata(metadata)
* .then((data) => {
* const hmacKeyMetadata = data[0];
* console.log(hmacKeyMetadata);
* });
* ```
*/
setMetadata: {
reqOpts: {
method: 'PUT',
},
},
};
const projectId = (options && options.projectId) || storage.projectId;
super({
parent: storage,
id: accessId,
baseUrl: `/projects/${projectId}/hmacKeys`,
methods,
});
this.storage = storage;
this.instanceRetryValue = storage.retryOptions.autoRetry;
}
setMetadata(metadata, optionsOrCallback, cb) {
// ETag preconditions are not currently supported. Retries should be disabled if the idempotency strategy is not set to RetryAlways
if (this.storage.retryOptions.idempotencyStrategy !==
IdempotencyStrategy.RetryAlways) {
this.storage.retryOptions.autoRetry = false;
}
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : {};
cb =
typeof optionsOrCallback === 'function'
? optionsOrCallback
: cb;
super
.setMetadata(metadata, options)
.then(resp => cb(null, ...resp))
.catch(cb)
.finally(() => {
this.storage.retryOptions.autoRetry = this.instanceRetryValue;
});
}
}
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
promisifyAll(HmacKey);

View File

@@ -0,0 +1,117 @@
import { Bucket } from './bucket.js';
export interface GetPolicyOptions {
userProject?: string;
requestedPolicyVersion?: number;
}
export type GetPolicyResponse = [Policy, unknown];
/**
* @callback GetPolicyCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The policy.
* @param {object} apiResponse The full API response.
*/
export interface GetPolicyCallback {
(err?: Error | null, acl?: Policy, apiResponse?: unknown): void;
}
/**
* @typedef {object} SetPolicyOptions
* @param {string} [userProject] The ID of the project which will be
* billed for the request.
*/
export interface SetPolicyOptions {
userProject?: string;
}
/**
* @typedef {array} SetPolicyResponse
* @property {object} 0 The policy.
* @property {object} 1 The full API response.
*/
export type SetPolicyResponse = [Policy, unknown];
/**
* @callback SetPolicyCallback
* @param {?Error} err Request error, if any.
* @param {object} acl The policy.
* @param {object} apiResponse The full API response.
*/
export interface SetPolicyCallback {
(err?: Error | null, acl?: Policy, apiResponse?: object): void;
}
export interface Policy {
bindings: PolicyBinding[];
version?: number;
etag?: string;
}
export interface PolicyBinding {
role: string;
members: string[];
condition?: Expr;
}
export interface Expr {
title?: string;
description?: string;
expression: string;
}
/**
* @typedef {array} TestIamPermissionsResponse
* @property {object} 0 A subset of permissions that the caller is allowed.
* @property {object} 1 The full API response.
*/
export type TestIamPermissionsResponse = [{
[key: string]: boolean;
}, unknown];
/**
* @callback TestIamPermissionsCallback
* @param {?Error} err Request error, if any.
* @param {object} acl A subset of permissions that the caller is allowed.
* @param {object} apiResponse The full API response.
*/
export interface TestIamPermissionsCallback {
(err?: Error | null, acl?: {
[key: string]: boolean;
} | null, apiResponse?: unknown): void;
}
/**
* @typedef {object} TestIamPermissionsOptions Configuration options for Iam#testPermissions().
* @param {string} [userProject] The ID of the project which will be
* billed for the request.
*/
export interface TestIamPermissionsOptions {
userProject?: string;
}
export declare enum IAMExceptionMessages {
POLICY_OBJECT_REQUIRED = "A policy object is required.",
PERMISSIONS_REQUIRED = "Permissions are required."
}
/**
* Get and set IAM policies for your Cloud Storage bucket.
*
* See {@link https://cloud.google.com/storage/docs/access-control/iam#short_title_iam_management| Cloud Storage IAM Management}
* See {@link https://cloud.google.com/iam/docs/granting-changing-revoking-access| Granting, Changing, and Revoking Access}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @constructor Iam
*
* @param {Bucket} bucket The parent instance.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* // bucket.iam
* ```
*/
declare class Iam {
private request_;
private resourceId_;
constructor(bucket: Bucket);
getPolicy(options?: GetPolicyOptions): Promise<GetPolicyResponse>;
getPolicy(options: GetPolicyOptions, callback: GetPolicyCallback): void;
getPolicy(callback: GetPolicyCallback): void;
setPolicy(policy: Policy, options?: SetPolicyOptions): Promise<SetPolicyResponse>;
setPolicy(policy: Policy, callback: SetPolicyCallback): void;
setPolicy(policy: Policy, options: SetPolicyOptions, callback: SetPolicyCallback): void;
testPermissions(permissions: string | string[], options?: TestIamPermissionsOptions): Promise<TestIamPermissionsResponse>;
testPermissions(permissions: string | string[], callback: TestIamPermissionsCallback): void;
testPermissions(permissions: string | string[], options: TestIamPermissionsOptions, callback: TestIamPermissionsCallback): void;
}
export { Iam };

View File

@@ -0,0 +1,303 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { promisifyAll } from '@google-cloud/promisify';
import { normalize } from './util.js';
export var IAMExceptionMessages;
(function (IAMExceptionMessages) {
IAMExceptionMessages["POLICY_OBJECT_REQUIRED"] = "A policy object is required.";
IAMExceptionMessages["PERMISSIONS_REQUIRED"] = "Permissions are required.";
})(IAMExceptionMessages || (IAMExceptionMessages = {}));
/**
* Get and set IAM policies for your Cloud Storage bucket.
*
* See {@link https://cloud.google.com/storage/docs/access-control/iam#short_title_iam_management| Cloud Storage IAM Management}
* See {@link https://cloud.google.com/iam/docs/granting-changing-revoking-access| Granting, Changing, and Revoking Access}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @constructor Iam
*
* @param {Bucket} bucket The parent instance.
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* // bucket.iam
* ```
*/
class Iam {
constructor(bucket) {
this.request_ = bucket.request.bind(bucket);
this.resourceId_ = 'buckets/' + bucket.getId();
}
/**
* @typedef {object} GetPolicyOptions Requested options for IAM#getPolicy().
* @property {number} [requestedPolicyVersion] The version of IAM policies to
* request. If a policy with a condition is requested without setting
* this, the server will return an error. This must be set to a value
* of 3 to retrieve IAM policies containing conditions. This is to
* prevent client code that isn't aware of IAM conditions from
* interpreting and modifying policies incorrectly. The service might
* return a policy with version lower than the one that was requested,
* based on the feature syntax in the policy fetched.
* See {@link https://cloud.google.com/iam/docs/policies#versions| IAM Policy versions}
* @property {string} [userProject] The ID of the project which will be
* billed for the request.
*/
/**
* @typedef {array} GetPolicyResponse
* @property {Policy} 0 The policy.
* @property {object} 1 The full API response.
*/
/**
* @typedef {object} Policy
* @property {PolicyBinding[]} policy.bindings Bindings associate members with roles.
* @property {string} [policy.etag] Etags are used to perform a read-modify-write.
* @property {number} [policy.version] The syntax schema version of the Policy.
* To set an IAM policy with conditional binding, this field must be set to
* 3 or greater.
* See {@link https://cloud.google.com/iam/docs/policies#versions| IAM Policy versions}
*/
/**
* @typedef {object} PolicyBinding
* @property {string} role Role that is assigned to members.
* @property {string[]} members Specifies the identities requesting access for the bucket.
* @property {Expr} [condition] The condition that is associated with this binding.
*/
/**
* @typedef {object} Expr
* @property {string} [title] An optional title for the expression, i.e. a
* short string describing its purpose. This can be used e.g. in UIs
* which allow to enter the expression.
* @property {string} [description] An optional description of the
* expression. This is a longer text which describes the expression,
* e.g. when hovered over it in a UI.
* @property {string} expression Textual representation of an expression in
* Common Expression Language syntax. The application context of the
* containing message determines which well-known feature set of CEL
* is supported.The condition that is associated with this binding.
*
* @see [Condition] https://cloud.google.com/storage/docs/access-control/iam#conditions
*/
/**
* Get the IAM policy.
*
* @param {GetPolicyOptions} [options] Request options.
* @param {GetPolicyCallback} [callback] Callback function.
* @returns {Promise<GetPolicyResponse>}
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/getIamPolicy| Buckets: setIamPolicy API Documentation}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* bucket.iam.getPolicy(
* {requestedPolicyVersion: 3},
* function(err, policy, apiResponse) {
*
* },
* );
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.getPolicy({requestedPolicyVersion: 3})
* .then(function(data) {
* const policy = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_view_bucket_iam_members
* Example of retrieving a bucket's IAM policy:
*/
getPolicy(optionsOrCallback, callback) {
const { options, callback: cb } = normalize(optionsOrCallback, callback);
const qs = {};
if (options.userProject) {
qs.userProject = options.userProject;
}
if (options.requestedPolicyVersion !== null &&
options.requestedPolicyVersion !== undefined) {
qs.optionsRequestedPolicyVersion = options.requestedPolicyVersion;
}
this.request_({
uri: '/iam',
qs,
}, cb);
}
/**
* Set the IAM policy.
*
* @throws {Error} If no policy is provided.
*
* @param {Policy} policy The policy.
* @param {SetPolicyOptions} [options] Configuration options.
* @param {SetPolicyCallback} callback Callback function.
* @returns {Promise<SetPolicyResponse>}
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/setIamPolicy| Buckets: setIamPolicy API Documentation}
* See {@link https://cloud.google.com/iam/docs/understanding-roles| IAM Roles}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* const myPolicy = {
* bindings: [
* {
* role: 'roles/storage.admin',
* members:
* ['serviceAccount:myotherproject@appspot.gserviceaccount.com']
* }
* ]
* };
*
* bucket.iam.setPolicy(myPolicy, function(err, policy, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.setPolicy(myPolicy).then(function(data) {
* const policy = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_add_bucket_iam_member
* Example of adding to a bucket's IAM policy:
*
* @example <caption>include:samples/iam.js</caption>
* region_tag:storage_remove_bucket_iam_member
* Example of removing from a bucket's IAM policy:
*/
setPolicy(policy, optionsOrCallback, callback) {
if (policy === null || typeof policy !== 'object') {
throw new Error(IAMExceptionMessages.POLICY_OBJECT_REQUIRED);
}
const { options, callback: cb } = normalize(optionsOrCallback, callback);
let maxRetries;
if (policy.etag === undefined) {
maxRetries = 0;
}
this.request_({
method: 'PUT',
uri: '/iam',
maxRetries,
json: Object.assign({
resourceId: this.resourceId_,
}, policy),
qs: options,
}, cb);
}
/**
* Test a set of permissions for a resource.
*
* @throws {Error} If permissions are not provided.
*
* @param {string|string[]} permissions The permission(s) to test for.
* @param {TestIamPermissionsOptions} [options] Configuration object.
* @param {TestIamPermissionsCallback} [callback] Callback function.
* @returns {Promise<TestIamPermissionsResponse>}
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/buckets/testIamPermissions| Buckets: testIamPermissions API Documentation}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
*
* //-
* // Test a single permission.
* //-
* const test = 'storage.buckets.delete';
*
* bucket.iam.testPermissions(test, function(err, permissions, apiResponse) {
* console.log(permissions);
* // {
* // "storage.buckets.delete": true
* // }
* });
*
* //-
* // Test several permissions at once.
* //-
* const tests = [
* 'storage.buckets.delete',
* 'storage.buckets.get'
* ];
*
* bucket.iam.testPermissions(tests, function(err, permissions) {
* console.log(permissions);
* // {
* // "storage.buckets.delete": false,
* // "storage.buckets.get": true
* // }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* bucket.iam.testPermissions(test).then(function(data) {
* const permissions = data[0];
* const apiResponse = data[1];
* });
* ```
*/
testPermissions(permissions, optionsOrCallback, callback) {
if (!Array.isArray(permissions) && typeof permissions !== 'string') {
throw new Error(IAMExceptionMessages.PERMISSIONS_REQUIRED);
}
const { options, callback: cb } = normalize(optionsOrCallback, callback);
const permissionsArray = Array.isArray(permissions)
? permissions
: [permissions];
const req = Object.assign({
permissions: permissionsArray,
}, options);
this.request_({
uri: '/iam/testPermissions',
qs: req,
useQuerystring: true,
}, (err, resp) => {
if (err) {
cb(err, null, resp);
return;
}
const availablePermissions = Array.isArray(resp.permissions)
? resp.permissions
: [];
const permissionsHash = permissionsArray.reduce((acc, permission) => {
acc[permission] = availablePermissions.indexOf(permission) > -1;
return acc;
}, {});
cb(null, permissionsHash, resp);
});
}
}
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
promisifyAll(Iam);
export { Iam };

View File

@@ -0,0 +1,57 @@
/**
* The `@google-cloud/storage` package has a single named export which is the
* {@link Storage} (ES6) class, which should be instantiated with `new`.
*
* See {@link Storage} and {@link ClientConfig} for client methods and
* configuration options.
*
* @module {Storage} @google-cloud/storage
* @alias nodejs-storage
*
* @example
* Install the client library with <a href="https://www.npmjs.com/">npm</a>:
* ```
* npm install --save @google-cloud/storage
* ```
*
* @example
* Import the client library
* ```
* const {Storage} = require('@google-cloud/storage');
* ```
*
* @example
* Create a client that uses <a
* href="https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application">Application
* Default Credentials (ADC)</a>:
* ```
* const storage = new Storage();
* ```
*
* @example
* Create a client with <a
* href="https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually">explicit
* credentials</a>:
* ```
* const storage = new Storage({ projectId:
* 'your-project-id', keyFilename: '/path/to/keyfile.json'
* });
* ```
*
* @example <caption>include:samples/quickstart.js</caption>
* region_tag:storage_quickstart
* Full quickstart example:
*/
export { ApiError } from './nodejs-common/index.js';
export { BucketCallback, BucketOptions, CreateBucketQuery, CreateBucketRequest, CreateBucketResponse, CreateHmacKeyCallback, CreateHmacKeyOptions, CreateHmacKeyResponse, GetBucketsCallback, GetBucketsRequest, GetBucketsResponse, GetHmacKeysCallback, GetHmacKeysOptions, GetHmacKeysResponse, GetServiceAccountCallback, GetServiceAccountOptions, GetServiceAccountResponse, HmacKeyResourceResponse, IdempotencyStrategy, PreconditionOptions, RETRYABLE_ERR_FN_DEFAULT, ServiceAccount, Storage, StorageOptions, } from './storage.js';
export { AclMetadata, AccessControlObject, AclOptions, AddAclCallback, AddAclOptions, AddAclResponse, GetAclCallback, GetAclOptions, GetAclResponse, RemoveAclCallback, RemoveAclOptions, RemoveAclResponse, UpdateAclCallback, UpdateAclOptions, UpdateAclResponse, } from './acl.js';
export { Bucket, BucketExistsCallback, BucketExistsOptions, BucketExistsResponse, BucketLockCallback, BucketLockResponse, BucketMetadata, CombineCallback, CombineOptions, CombineResponse, CreateChannelCallback, CreateChannelConfig, CreateChannelOptions, CreateChannelResponse, CreateNotificationCallback, CreateNotificationOptions, CreateNotificationResponse, DeleteBucketCallback, DeleteBucketOptions, DeleteBucketResponse, DeleteFilesCallback, DeleteFilesOptions, DeleteLabelsCallback, DeleteLabelsResponse, DisableRequesterPaysCallback, DisableRequesterPaysResponse, EnableRequesterPaysCallback, EnableRequesterPaysResponse, GetBucketCallback, GetBucketMetadataCallback, GetBucketMetadataOptions, GetBucketMetadataResponse, GetBucketOptions, GetBucketResponse, GetBucketSignedUrlConfig, GetFilesCallback, GetFilesOptions, GetFilesResponse, GetLabelsCallback, GetLabelsOptions, GetLabelsResponse, GetNotificationsCallback, GetNotificationsOptions, GetNotificationsResponse, Labels, LifecycleAction, LifecycleCondition, LifecycleRule, MakeBucketPrivateCallback, MakeBucketPrivateOptions, MakeBucketPrivateResponse, MakeBucketPublicCallback, MakeBucketPublicOptions, MakeBucketPublicResponse, SetBucketMetadataCallback, SetBucketMetadataOptions, SetBucketMetadataResponse, SetBucketStorageClassCallback, SetBucketStorageClassOptions, SetLabelsCallback, SetLabelsOptions, SetLabelsResponse, UploadCallback, UploadOptions, UploadResponse, } from './bucket.js';
export * from './crc32c.js';
export { Channel, StopCallback } from './channel.js';
export { CopyCallback, CopyOptions, CopyResponse, CreateReadStreamOptions, CreateResumableUploadCallback, CreateResumableUploadOptions, CreateResumableUploadResponse, CreateWriteStreamOptions, DeleteFileCallback, DeleteFileOptions, DeleteFileResponse, DownloadCallback, DownloadOptions, DownloadResponse, EncryptionKeyOptions, File, FileExistsCallback, FileExistsOptions, FileExistsResponse, FileMetadata, FileOptions, GetExpirationDateCallback, GetExpirationDateResponse, GetFileCallback, GetFileMetadataCallback, GetFileMetadataOptions, GetFileMetadataResponse, GetFileOptions, GetFileResponse, GenerateSignedPostPolicyV2Callback, GenerateSignedPostPolicyV2Options, GenerateSignedPostPolicyV2Response, GenerateSignedPostPolicyV4Callback, GenerateSignedPostPolicyV4Options, GenerateSignedPostPolicyV4Response, GetSignedUrlConfig, MakeFilePrivateCallback, MakeFilePrivateOptions, MakeFilePrivateResponse, MakeFilePublicCallback, MakeFilePublicResponse, MoveCallback, MoveOptions, MoveResponse, MoveFileAtomicOptions, MoveFileAtomicCallback, MoveFileAtomicResponse, PolicyDocument, PolicyFields, PredefinedAcl, RotateEncryptionKeyCallback, RotateEncryptionKeyOptions, RotateEncryptionKeyResponse, SaveCallback, SaveData, SaveOptions, SetFileMetadataCallback, SetFileMetadataOptions, SetFileMetadataResponse, SetStorageClassCallback, SetStorageClassOptions, SetStorageClassResponse, SignedPostPolicyV4Output, } from './file.js';
export * from './hash-stream-validator.js';
export { HmacKey, HmacKeyMetadata, HmacKeyMetadataCallback, HmacKeyMetadataResponse, SetHmacKeyMetadata, SetHmacKeyMetadataOptions, } from './hmacKey.js';
export { GetPolicyCallback, GetPolicyOptions, GetPolicyResponse, Iam, Policy, SetPolicyCallback, SetPolicyOptions, SetPolicyResponse, TestIamPermissionsCallback, TestIamPermissionsOptions, TestIamPermissionsResponse, } from './iam.js';
export { DeleteNotificationCallback, DeleteNotificationOptions, GetNotificationCallback, GetNotificationMetadataCallback, GetNotificationMetadataOptions, GetNotificationMetadataResponse, GetNotificationOptions, GetNotificationResponse, Notification, NotificationMetadata, } from './notification.js';
export { GetSignedUrlCallback, GetSignedUrlResponse } from './signer.js';
export * from './transfer-manager.js';

View File

@@ -0,0 +1,68 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/**
* The `@google-cloud/storage` package has a single named export which is the
* {@link Storage} (ES6) class, which should be instantiated with `new`.
*
* See {@link Storage} and {@link ClientConfig} for client methods and
* configuration options.
*
* @module {Storage} @google-cloud/storage
* @alias nodejs-storage
*
* @example
* Install the client library with <a href="https://www.npmjs.com/">npm</a>:
* ```
* npm install --save @google-cloud/storage
* ```
*
* @example
* Import the client library
* ```
* const {Storage} = require('@google-cloud/storage');
* ```
*
* @example
* Create a client that uses <a
* href="https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application">Application
* Default Credentials (ADC)</a>:
* ```
* const storage = new Storage();
* ```
*
* @example
* Create a client with <a
* href="https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually">explicit
* credentials</a>:
* ```
* const storage = new Storage({ projectId:
* 'your-project-id', keyFilename: '/path/to/keyfile.json'
* });
* ```
*
* @example <caption>include:samples/quickstart.js</caption>
* region_tag:storage_quickstart
* Full quickstart example:
*/
export { ApiError } from './nodejs-common/index.js';
export { IdempotencyStrategy, RETRYABLE_ERR_FN_DEFAULT, Storage, } from './storage.js';
export { Bucket, } from './bucket.js';
export * from './crc32c.js';
export { Channel } from './channel.js';
export { File, } from './file.js';
export * from './hash-stream-validator.js';
export { HmacKey, } from './hmacKey.js';
export { Iam, } from './iam.js';
export { Notification, } from './notification.js';
export * from './transfer-manager.js';

View File

@@ -0,0 +1,19 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
export { GoogleAuthOptions } from 'google-auth-library';
export { Service, ServiceConfig, ServiceOptions, StreamRequestOptions, } from './service.js';
export { BaseMetadata, DeleteCallback, ExistsCallback, GetConfig, InstanceResponseCallback, Interceptor, MetadataCallback, MetadataResponse, Methods, ResponseCallback, ServiceObject, ServiceObjectConfig, ServiceObjectParent, SetMetadataResponse, } from './service-object.js';
export { Abortable, AbortableDuplex, ApiError, BodyResponseCallback, DecorateRequestOptions, ResponseBody, util, } from './util.js';

View File

@@ -0,0 +1,3 @@
export { Service, } from './service.js';
export { ServiceObject, } from './service-object.js';
export { ApiError, util, } from './util.js';

View File

@@ -0,0 +1,217 @@
import { EventEmitter } from 'events';
import * as r from 'teeny-request';
import { ApiError, BodyResponseCallback, DecorateRequestOptions } from './util.js';
export type RequestResponse = [unknown, r.Response];
export interface ServiceObjectParent {
interceptors: Interceptor[];
getRequestInterceptors(): Function[];
requestStream(reqOpts: DecorateRequestOptions): r.Request;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
}
export interface Interceptor {
request(opts: r.Options): DecorateRequestOptions;
}
export type GetMetadataOptions = object;
export type MetadataResponse<K> = [K, r.Response];
export type MetadataCallback<K> = (err: Error | null, metadata?: K, apiResponse?: r.Response) => void;
export type ExistsOptions = object;
export interface ExistsCallback {
(err: Error | null, exists?: boolean): void;
}
export interface ServiceObjectConfig {
/**
* The base URL to make API requests to.
*/
baseUrl?: string;
/**
* The method which creates this object.
*/
createMethod?: Function;
/**
* The identifier of the object. For example, the name of a Storage bucket or
* Pub/Sub topic.
*/
id?: string;
/**
* A map of each method name that should be inherited.
*/
methods?: Methods;
/**
* The parent service instance. For example, an instance of Storage if the
* object is Bucket.
*/
parent: ServiceObjectParent;
/**
* Override of projectId, used to allow access to resources in another project.
* For example, a BigQuery dataset in another project to which the user has been
* granted permission.
*/
projectId?: string;
}
export interface Methods {
[methodName: string]: {
reqOpts?: r.CoreOptions;
} | boolean;
}
export interface InstanceResponseCallback<T> {
(err: ApiError | null, instance?: T | null, apiResponse?: r.Response): void;
}
export interface CreateOptions {
}
export type CreateResponse<T> = any[];
export interface CreateCallback<T> {
(err: ApiError | null, instance?: T | null, ...args: any[]): void;
}
export type DeleteOptions = {
ignoreNotFound?: boolean;
ifGenerationMatch?: number | string;
ifGenerationNotMatch?: number | string;
ifMetagenerationMatch?: number | string;
ifMetagenerationNotMatch?: number | string;
} & object;
export interface DeleteCallback {
(err: Error | null, apiResponse?: r.Response): void;
}
export interface GetConfig {
/**
* Create the object if it doesn't already exist.
*/
autoCreate?: boolean;
}
export type GetOrCreateOptions = GetConfig & CreateOptions;
export type GetResponse<T> = [T, r.Response];
export interface ResponseCallback {
(err?: Error | null, apiResponse?: r.Response): void;
}
export type SetMetadataResponse<K> = [K];
export type SetMetadataOptions = object;
export interface BaseMetadata {
id?: string;
kind?: string;
etag?: string;
selfLink?: string;
[key: string]: unknown;
}
/**
* ServiceObject is a base class, meant to be inherited from by a "service
* object," like a BigQuery dataset or Storage bucket.
*
* Most of the time, these objects share common functionality; they can be
* created or deleted, and you can get or set their metadata.
*
* By inheriting from this class, a service object will be extended with these
* shared behaviors. Note that any method can be overridden when the service
* object requires specific behavior.
*/
declare class ServiceObject<T, K extends BaseMetadata> extends EventEmitter {
metadata: K;
baseUrl?: string;
parent: ServiceObjectParent;
id?: string;
private createMethod?;
protected methods: Methods;
interceptors: Interceptor[];
projectId?: string;
constructor(config: ServiceObjectConfig);
/**
* Create the object.
*
* @param {object=} options - Configuration object.
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.instance - The instance.
* @param {object} callback.apiResponse - The full API response.
*/
create(options?: CreateOptions): Promise<CreateResponse<T>>;
create(options: CreateOptions, callback: CreateCallback<T>): void;
create(callback: CreateCallback<T>): void;
/**
* Delete the object.
*
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
delete(options?: DeleteOptions): Promise<[r.Response]>;
delete(options: DeleteOptions, callback: DeleteCallback): void;
delete(callback: DeleteCallback): void;
/**
* Check if the object exists.
*
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {boolean} callback.exists - Whether the object exists or not.
*/
exists(options?: ExistsOptions): Promise<[boolean]>;
exists(options: ExistsOptions, callback: ExistsCallback): void;
exists(callback: ExistsCallback): void;
/**
* Get the object if it exists. Optionally have the object created if an
* options object is provided with `autoCreate: true`.
*
* @param {object=} options - The configuration object that will be used to
* create the object if necessary.
* @param {boolean} options.autoCreate - Create the object if it doesn't already exist.
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.instance - The instance.
* @param {object} callback.apiResponse - The full API response.
*/
get(options?: GetOrCreateOptions): Promise<GetResponse<T>>;
get(callback: InstanceResponseCallback<T>): void;
get(options: GetOrCreateOptions, callback: InstanceResponseCallback<T>): void;
/**
* Get the metadata of this object.
*
* @param {function} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.metadata - The metadata for this object.
* @param {object} callback.apiResponse - The full API response.
*/
getMetadata(options?: GetMetadataOptions): Promise<MetadataResponse<K>>;
getMetadata(options: GetMetadataOptions, callback: MetadataCallback<K>): void;
getMetadata(callback: MetadataCallback<K>): void;
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors(): Function[];
/**
* Set the metadata for this object.
*
* @param {object} metadata - The metadata to set on this object.
* @param {object=} options - Configuration options.
* @param {function=} callback - The callback function.
* @param {?error} callback.err - An error returned while making this request.
* @param {object} callback.apiResponse - The full API response.
*/
setMetadata(metadata: K, options?: SetMetadataOptions): Promise<SetMetadataResponse<K>>;
setMetadata(metadata: K, callback: MetadataCallback<K>): void;
setMetadata(metadata: K, options: SetMetadataOptions, callback: MetadataCallback<K>): void;
/**
* Make an authenticated API request.
*
* @private
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
private request_;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
request(reqOpts: DecorateRequestOptions): Promise<RequestResponse>;
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts: DecorateRequestOptions): r.Request;
}
export { ServiceObject };

View File

@@ -0,0 +1,289 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { promisifyAll } from '@google-cloud/promisify';
import { EventEmitter } from 'events';
import { util, } from './util.js';
/**
* ServiceObject is a base class, meant to be inherited from by a "service
* object," like a BigQuery dataset or Storage bucket.
*
* Most of the time, these objects share common functionality; they can be
* created or deleted, and you can get or set their metadata.
*
* By inheriting from this class, a service object will be extended with these
* shared behaviors. Note that any method can be overridden when the service
* object requires specific behavior.
*/
// eslint-disable-next-line @typescript-eslint/no-explicit-any
class ServiceObject extends EventEmitter {
/*
* @constructor
* @alias module:common/service-object
*
* @private
*
* @param {object} config - Configuration object.
* @param {string} config.baseUrl - The base URL to make API requests to.
* @param {string} config.createMethod - The method which creates this object.
* @param {string=} config.id - The identifier of the object. For example, the
* name of a Storage bucket or Pub/Sub topic.
* @param {object=} config.methods - A map of each method name that should be inherited.
* @param {object} config.methods[].reqOpts - Default request options for this
* particular method. A common use case is when `setMetadata` requires a
* `PUT` method to override the default `PATCH`.
* @param {object} config.parent - The parent service instance. For example, an
* instance of Storage if the object is Bucket.
*/
constructor(config) {
super();
this.metadata = {};
this.baseUrl = config.baseUrl;
this.parent = config.parent; // Parent class.
this.id = config.id; // Name or ID (e.g. dataset ID, bucket name, etc).
this.createMethod = config.createMethod;
this.methods = config.methods || {};
this.interceptors = [];
this.projectId = config.projectId;
if (config.methods) {
// This filters the ServiceObject instance (e.g. a "File") to only have
// the configured methods. We make a couple of exceptions for core-
// functionality ("request()" and "getRequestInterceptors()")
Object.getOwnPropertyNames(ServiceObject.prototype)
.filter(methodName => {
return (
// All ServiceObjects need `request` and `getRequestInterceptors`.
// clang-format off
!/^request/.test(methodName) &&
!/^getRequestInterceptors/.test(methodName) &&
// clang-format on
// The ServiceObject didn't redefine the method.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this[methodName] ===
// eslint-disable-next-line @typescript-eslint/no-explicit-any
ServiceObject.prototype[methodName] &&
// This method isn't wanted.
!config.methods[methodName]);
})
.forEach(methodName => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this[methodName] = undefined;
});
}
}
create(optionsOrCallback, callback) {
// eslint-disable-next-line @typescript-eslint/no-this-alias
const self = this;
const args = [this.id];
if (typeof optionsOrCallback === 'function') {
callback = optionsOrCallback;
}
if (typeof optionsOrCallback === 'object') {
args.push(optionsOrCallback);
}
// Wrap the callback to return *this* instance of the object, not the
// newly-created one.
// tslint: disable-next-line no-any
function onCreate(...args) {
const [err, instance] = args;
if (!err) {
self.metadata = instance.metadata;
if (self.id && instance.metadata) {
self.id = instance.metadata.id;
}
args[1] = self; // replace the created `instance` with this one.
}
callback(...args);
}
args.push(onCreate);
// eslint-disable-next-line prefer-spread
this.createMethod.apply(null, args);
}
delete(optionsOrCallback, cb) {
var _a;
const [options, callback] = util.maybeOptionsOrCallback(optionsOrCallback, cb);
const ignoreNotFound = options.ignoreNotFound;
delete options.ignoreNotFound;
const methodConfig = (typeof this.methods.delete === 'object' && this.methods.delete) || {};
const reqOpts = {
method: 'DELETE',
uri: '',
...methodConfig.reqOpts,
qs: {
...(_a = methodConfig.reqOpts) === null || _a === void 0 ? void 0 : _a.qs,
...options,
},
};
// The `request` method may have been overridden to hold any special
// behavior. Ensure we call the original `request` method.
ServiceObject.prototype.request.call(this, reqOpts, (err, body, res) => {
if (err) {
if (err.code === 404 && ignoreNotFound) {
err = null;
}
}
callback(err, res);
});
}
exists(optionsOrCallback, cb) {
const [options, callback] = util.maybeOptionsOrCallback(optionsOrCallback, cb);
this.get(options, err => {
if (err) {
if (err.code === 404) {
callback(null, false);
}
else {
callback(err);
}
return;
}
callback(null, true);
});
}
get(optionsOrCallback, cb) {
// eslint-disable-next-line @typescript-eslint/no-this-alias
const self = this;
const [opts, callback] = util.maybeOptionsOrCallback(optionsOrCallback, cb);
const options = Object.assign({}, opts);
const autoCreate = options.autoCreate && typeof this.create === 'function';
delete options.autoCreate;
function onCreate(err, instance, apiResponse) {
if (err) {
if (err.code === 409) {
self.get(options, callback);
return;
}
callback(err, null, apiResponse);
return;
}
callback(null, instance, apiResponse);
}
this.getMetadata(options, (err, metadata) => {
if (err) {
if (err.code === 404 && autoCreate) {
const args = [];
if (Object.keys(options).length > 0) {
args.push(options);
}
args.push(onCreate);
self.create(...args);
return;
}
callback(err, null, metadata);
return;
}
callback(null, self, metadata);
});
}
getMetadata(optionsOrCallback, cb) {
var _a;
const [options, callback] = util.maybeOptionsOrCallback(optionsOrCallback, cb);
const methodConfig = (typeof this.methods.getMetadata === 'object' &&
this.methods.getMetadata) ||
{};
const reqOpts = {
uri: '',
...methodConfig.reqOpts,
qs: {
...(_a = methodConfig.reqOpts) === null || _a === void 0 ? void 0 : _a.qs,
...options,
},
};
// The `request` method may have been overridden to hold any special
// behavior. Ensure we call the original `request` method.
ServiceObject.prototype.request.call(this, reqOpts, (err, body, res) => {
this.metadata = body;
callback(err, this.metadata, res);
});
}
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors() {
// Interceptors should be returned in the order they were assigned.
const localInterceptors = this.interceptors
.filter(interceptor => typeof interceptor.request === 'function')
.map(interceptor => interceptor.request);
return this.parent.getRequestInterceptors().concat(localInterceptors);
}
setMetadata(metadata, optionsOrCallback, cb) {
var _a, _b;
const [options, callback] = util.maybeOptionsOrCallback(optionsOrCallback, cb);
const methodConfig = (typeof this.methods.setMetadata === 'object' &&
this.methods.setMetadata) ||
{};
const reqOpts = {
method: 'PATCH',
uri: '',
...methodConfig.reqOpts,
json: {
...(_a = methodConfig.reqOpts) === null || _a === void 0 ? void 0 : _a.json,
...metadata,
},
qs: {
...(_b = methodConfig.reqOpts) === null || _b === void 0 ? void 0 : _b.qs,
...options,
},
};
// The `request` method may have been overridden to hold any special
// behavior. Ensure we call the original `request` method.
ServiceObject.prototype.request.call(this, reqOpts, (err, body, res) => {
this.metadata = body;
callback(err, this.metadata, res);
});
}
request_(reqOpts, callback) {
reqOpts = { ...reqOpts };
if (this.projectId) {
reqOpts.projectId = this.projectId;
}
const isAbsoluteUrl = reqOpts.uri.indexOf('http') === 0;
const uriComponents = [this.baseUrl, this.id || '', reqOpts.uri];
if (isAbsoluteUrl) {
uriComponents.splice(0, uriComponents.indexOf(reqOpts.uri));
}
reqOpts.uri = uriComponents
.filter(x => x.trim()) // Limit to non-empty strings.
.map(uriComponent => {
const trimSlashesRegex = /^\/*|\/*$/g;
return uriComponent.replace(trimSlashesRegex, '');
})
.join('/');
const childInterceptors = Array.isArray(reqOpts.interceptors_)
? reqOpts.interceptors_
: [];
const localInterceptors = [].slice.call(this.interceptors);
reqOpts.interceptors_ = childInterceptors.concat(localInterceptors);
if (reqOpts.shouldReturnStream) {
return this.parent.requestStream(reqOpts);
}
this.parent.request(reqOpts, callback);
}
request(reqOpts, callback) {
this.request_(reqOpts, callback);
}
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts) {
const opts = { ...reqOpts, shouldReturnStream: true };
return this.request_(opts);
}
}
promisifyAll(ServiceObject, { exclude: ['getRequestInterceptors'] });
export { ServiceObject };

View File

@@ -0,0 +1,125 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { AuthClient, GoogleAuth, GoogleAuthOptions } from 'google-auth-library';
import * as r from 'teeny-request';
import { Interceptor } from './service-object.js';
import { BodyResponseCallback, DecorateRequestOptions, MakeAuthenticatedRequest, PackageJson } from './util.js';
export declare const DEFAULT_PROJECT_ID_TOKEN = "{{projectId}}";
export interface StreamRequestOptions extends DecorateRequestOptions {
shouldReturnStream: true;
}
export interface ServiceConfig {
/**
* The base URL to make API requests to.
*/
baseUrl: string;
/**
* The API Endpoint to use when connecting to the service.
* Example: storage.googleapis.com
*/
apiEndpoint: string;
/**
* The scopes required for the request.
*/
scopes: string[];
projectIdRequired?: boolean;
packageJson: PackageJson;
/**
* Reuse an existing `AuthClient` or `GoogleAuth` client instead of creating a new one.
*/
authClient?: AuthClient | GoogleAuth;
/**
* Set to true if the endpoint is a custom URL
*/
customEndpoint?: boolean;
}
export interface ServiceOptions extends Omit<GoogleAuthOptions, 'authClient'> {
authClient?: AuthClient | GoogleAuth;
interceptors_?: Interceptor[];
email?: string;
token?: string;
timeout?: number;
userAgent?: string;
useAuthWithCustomEndpoint?: boolean;
}
export declare class Service {
baseUrl: string;
private globalInterceptors;
interceptors: Interceptor[];
private packageJson;
projectId: string;
private projectIdRequired;
providedUserAgent?: string;
makeAuthenticatedRequest: MakeAuthenticatedRequest;
authClient: GoogleAuth<AuthClient>;
apiEndpoint: string;
timeout?: number;
universeDomain: string;
customEndpoint: boolean;
/**
* Service is a base class, meant to be inherited from by a "service," like
* BigQuery or Storage.
*
* This handles making authenticated requests by exposing a `makeReq_`
* function.
*
* @constructor
* @alias module:common/service
*
* @param {object} config - Configuration object.
* @param {string} config.baseUrl - The base URL to make API requests to.
* @param {string[]} config.scopes - The scopes required for the request.
* @param {object=} options - [Configuration object](#/docs).
*/
constructor(config: ServiceConfig, options?: ServiceOptions);
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors(): Function[];
/**
* Get and update the Service's project ID.
*
* @param {function} callback - The callback function.
*/
getProjectId(): Promise<string>;
getProjectId(callback: (err: Error | null, projectId?: string) => void): void;
protected getProjectIdAsync(): Promise<string>;
/**
* Make an authenticated API request.
*
* @private
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
private request_;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
request(reqOpts: DecorateRequestOptions, callback: BodyResponseCallback): void;
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts: DecorateRequestOptions): r.Request;
}

View File

@@ -0,0 +1,181 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { DEFAULT_UNIVERSE, } from 'google-auth-library';
import * as uuid from 'uuid';
import { GCCL_GCS_CMD_KEY, util, } from './util.js';
import { getRuntimeTrackingString, getUserAgentString, getModuleFormat, } from '../util.js';
export const DEFAULT_PROJECT_ID_TOKEN = '{{projectId}}';
export class Service {
/**
* Service is a base class, meant to be inherited from by a "service," like
* BigQuery or Storage.
*
* This handles making authenticated requests by exposing a `makeReq_`
* function.
*
* @constructor
* @alias module:common/service
*
* @param {object} config - Configuration object.
* @param {string} config.baseUrl - The base URL to make API requests to.
* @param {string[]} config.scopes - The scopes required for the request.
* @param {object=} options - [Configuration object](#/docs).
*/
constructor(config, options = {}) {
this.baseUrl = config.baseUrl;
this.apiEndpoint = config.apiEndpoint;
this.timeout = options.timeout;
this.globalInterceptors = Array.isArray(options.interceptors_)
? options.interceptors_
: [];
this.interceptors = [];
this.packageJson = config.packageJson;
this.projectId = options.projectId || DEFAULT_PROJECT_ID_TOKEN;
this.projectIdRequired = config.projectIdRequired !== false;
this.providedUserAgent = options.userAgent;
this.universeDomain = options.universeDomain || DEFAULT_UNIVERSE;
this.customEndpoint = config.customEndpoint || false;
this.makeAuthenticatedRequest = util.makeAuthenticatedRequestFactory({
...config,
projectIdRequired: this.projectIdRequired,
projectId: this.projectId,
authClient: options.authClient || config.authClient,
credentials: options.credentials,
keyFile: options.keyFilename,
email: options.email,
clientOptions: {
universeDomain: options.universeDomain,
...options.clientOptions,
},
});
this.authClient = this.makeAuthenticatedRequest.authClient;
const isCloudFunctionEnv = !!process.env.FUNCTION_NAME;
if (isCloudFunctionEnv) {
this.interceptors.push({
request(reqOpts) {
reqOpts.forever = false;
return reqOpts;
},
});
}
}
/**
* Return the user's custom request interceptors.
*/
getRequestInterceptors() {
// Interceptors should be returned in the order they were assigned.
return [].slice
.call(this.globalInterceptors)
.concat(this.interceptors)
.filter(interceptor => typeof interceptor.request === 'function')
.map(interceptor => interceptor.request);
}
getProjectId(callback) {
if (!callback) {
return this.getProjectIdAsync();
}
this.getProjectIdAsync().then(p => callback(null, p), callback);
}
async getProjectIdAsync() {
const projectId = await this.authClient.getProjectId();
if (this.projectId === DEFAULT_PROJECT_ID_TOKEN && projectId) {
this.projectId = projectId;
}
return this.projectId;
}
request_(reqOpts, callback) {
reqOpts = { ...reqOpts, timeout: this.timeout };
const isAbsoluteUrl = reqOpts.uri.indexOf('http') === 0;
const uriComponents = [this.baseUrl];
if (this.projectIdRequired) {
if (reqOpts.projectId) {
uriComponents.push('projects');
uriComponents.push(reqOpts.projectId);
}
else {
uriComponents.push('projects');
uriComponents.push(this.projectId);
}
}
uriComponents.push(reqOpts.uri);
if (isAbsoluteUrl) {
uriComponents.splice(0, uriComponents.indexOf(reqOpts.uri));
}
reqOpts.uri = uriComponents
.map(uriComponent => {
const trimSlashesRegex = /^\/*|\/*$/g;
return uriComponent.replace(trimSlashesRegex, '');
})
.join('/')
// Some URIs have colon separators.
// Bad: https://.../projects/:list
// Good: https://.../projects:list
.replace(/\/:/g, ':');
const requestInterceptors = this.getRequestInterceptors();
const interceptorArray = Array.isArray(reqOpts.interceptors_)
? reqOpts.interceptors_
: [];
interceptorArray.forEach(interceptor => {
if (typeof interceptor.request === 'function') {
requestInterceptors.push(interceptor.request);
}
});
requestInterceptors.forEach(requestInterceptor => {
reqOpts = requestInterceptor(reqOpts);
});
delete reqOpts.interceptors_;
const pkg = this.packageJson;
let userAgent = getUserAgentString();
if (this.providedUserAgent) {
userAgent = `${this.providedUserAgent} ${userAgent}`;
}
reqOpts.headers = {
...reqOpts.headers,
'User-Agent': userAgent,
'x-goog-api-client': `${getRuntimeTrackingString()} gccl/${pkg.version}-${getModuleFormat()} gccl-invocation-id/${uuid.v4()}`,
};
if (reqOpts[GCCL_GCS_CMD_KEY]) {
reqOpts.headers['x-goog-api-client'] +=
` gccl-gcs-cmd/${reqOpts[GCCL_GCS_CMD_KEY]}`;
}
if (reqOpts.shouldReturnStream) {
return this.makeAuthenticatedRequest(reqOpts);
}
else {
this.makeAuthenticatedRequest(reqOpts, callback);
}
}
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
* @param {function} callback - The callback function passed to `request`.
*/
request(reqOpts, callback) {
Service.prototype.request_.call(this, reqOpts, callback);
}
/**
* Make an authenticated API request.
*
* @param {object} reqOpts - Request options that are passed to `request`.
* @param {string} reqOpts.uri - A URI relative to the baseUrl.
*/
requestStream(reqOpts) {
const opts = { ...reqOpts, shouldReturnStream: true };
return Service.prototype.request_.call(this, opts);
}
}

View File

@@ -0,0 +1,333 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { AuthClient, GoogleAuth, GoogleAuthOptions } from 'google-auth-library';
import { CredentialBody } from 'google-auth-library';
import * as r from 'teeny-request';
import { Duplex, DuplexOptions, Readable, Writable } from 'stream';
import { Interceptor } from './service-object.js';
/**
* A unique symbol for providing a `gccl-gcs-cmd` value
* for the `X-Goog-API-Client` header.
*
* E.g. the `V` in `X-Goog-API-Client: gccl-gcs-cmd/V`
**/
export declare const GCCL_GCS_CMD_KEY: unique symbol;
export type ResponseBody = any;
export interface DuplexifyOptions extends DuplexOptions {
autoDestroy?: boolean;
end?: boolean;
}
export interface Duplexify extends Duplex {
readonly destroyed: boolean;
setWritable(writable: Writable | false | null): void;
setReadable(readable: Readable | false | null): void;
}
export interface DuplexifyConstructor {
obj(writable?: Writable | false | null, readable?: Readable | false | null, options?: DuplexifyOptions): Duplexify;
new (writable?: Writable | false | null, readable?: Readable | false | null, options?: DuplexifyOptions): Duplexify;
(writable?: Writable | false | null, readable?: Readable | false | null, options?: DuplexifyOptions): Duplexify;
}
export interface ParsedHttpRespMessage {
resp: r.Response;
err?: ApiError;
}
export interface MakeAuthenticatedRequest {
(reqOpts: DecorateRequestOptions): Duplexify;
(reqOpts: DecorateRequestOptions, options?: MakeAuthenticatedRequestOptions): void | Abortable;
(reqOpts: DecorateRequestOptions, callback?: BodyResponseCallback): void | Abortable;
(reqOpts: DecorateRequestOptions, optionsOrCallback?: MakeAuthenticatedRequestOptions | BodyResponseCallback): void | Abortable | Duplexify;
getCredentials: (callback: (err?: Error | null, credentials?: CredentialBody) => void) => void;
authClient: GoogleAuth<AuthClient>;
}
export interface Abortable {
abort(): void;
}
export type AbortableDuplex = Duplexify & Abortable;
export interface PackageJson {
name: string;
version: string;
}
export interface MakeAuthenticatedRequestFactoryConfig extends Omit<GoogleAuthOptions, 'authClient'> {
/**
* Automatically retry requests if the response is related to rate limits or
* certain intermittent server errors. We will exponentially backoff
* subsequent requests by default. (default: true)
*/
autoRetry?: boolean;
/**
* If true, just return the provided request options. Default: false.
*/
customEndpoint?: boolean;
/**
* If true, will authenticate when using a custom endpoint. Default: false.
*/
useAuthWithCustomEndpoint?: boolean;
/**
* Account email address, required for PEM/P12 usage.
*/
email?: string;
/**
* Maximum number of automatic retries attempted before returning the error.
* (default: 3)
*/
maxRetries?: number;
stream?: Duplexify;
/**
* A pre-instantiated `AuthClient` or `GoogleAuth` client that should be used.
* A new client will be created if this is not set.
*/
authClient?: AuthClient | GoogleAuth;
/**
* Determines if a projectId is required for authenticated requests. Defaults to `true`.
*/
projectIdRequired?: boolean;
}
export interface MakeAuthenticatedRequestOptions {
onAuthenticated: OnAuthenticatedCallback;
}
export interface OnAuthenticatedCallback {
(err: Error | null, reqOpts?: DecorateRequestOptions): void;
}
export interface GoogleErrorBody {
code: number;
errors?: GoogleInnerError[];
response: r.Response;
message?: string;
}
export interface GoogleInnerError {
reason?: string;
message?: string;
}
export interface MakeWritableStreamOptions {
/**
* A connection instance used to get a token with and send the request
* through.
*/
connection?: {};
/**
* Metadata to send at the head of the request.
*/
metadata?: {
contentType?: string;
};
/**
* Request object, in the format of a standard Node.js http.request() object.
*/
request?: r.Options;
makeAuthenticatedRequest(reqOpts: r.OptionsWithUri & {
[GCCL_GCS_CMD_KEY]?: string;
}, fnobj: {
onAuthenticated(err: Error | null, authenticatedReqOpts?: r.Options): void;
}): void;
}
export interface DecorateRequestOptions extends r.CoreOptions {
autoPaginate?: boolean;
autoPaginateVal?: boolean;
objectMode?: boolean;
maxRetries?: number;
uri: string;
interceptors_?: Interceptor[];
shouldReturnStream?: boolean;
projectId?: string;
[GCCL_GCS_CMD_KEY]?: string;
}
export interface ParsedHttpResponseBody {
body: ResponseBody;
err?: Error;
}
/**
* Custom error type for API errors.
*
* @param {object} errorBody - Error object.
*/
export declare class ApiError extends Error {
code?: number;
errors?: GoogleInnerError[];
response?: r.Response;
constructor(errorMessage: string);
constructor(errorBody: GoogleErrorBody);
/**
* Pieces together an error message by combining all unique error messages
* returned from a single GoogleError
*
* @private
*
* @param {GoogleErrorBody} err The original error.
* @param {GoogleInnerError[]} [errors] Inner errors, if any.
* @returns {string}
*/
static createMultiErrorMessage(err: GoogleErrorBody, errors?: GoogleInnerError[]): string;
}
/**
* Custom error type for partial errors returned from the API.
*
* @param {object} b - Error object.
*/
export declare class PartialFailureError extends Error {
errors?: GoogleInnerError[];
response?: r.Response;
constructor(b: GoogleErrorBody);
}
export interface BodyResponseCallback {
(err: Error | ApiError | null, body?: ResponseBody, res?: r.Response): void;
}
export interface RetryOptions {
retryDelayMultiplier?: number;
totalTimeout?: number;
maxRetryDelay?: number;
autoRetry?: boolean;
maxRetries?: number;
retryableErrorFn?: (err: ApiError) => boolean;
}
export interface MakeRequestConfig {
/**
* Automatically retry requests if the response is related to rate limits or
* certain intermittent server errors. We will exponentially backoff
* subsequent requests by default. (default: true)
*/
autoRetry?: boolean;
/**
* Maximum number of automatic retries attempted before returning the error.
* (default: 3)
*/
maxRetries?: number;
retries?: number;
retryOptions?: RetryOptions;
stream?: Duplexify;
shouldRetryFn?: (response?: r.Response) => boolean;
}
export declare class Util {
ApiError: typeof ApiError;
PartialFailureError: typeof PartialFailureError;
/**
* No op.
*
* @example
* function doSomething(callback) {
* callback = callback || noop;
* }
*/
noop(): void;
/**
* Uniformly process an API response.
*
* @param {*} err - Error value.
* @param {*} resp - Response value.
* @param {*} body - Body value.
* @param {function} callback - The callback function.
*/
handleResp(err: Error | null, resp?: r.Response | null, body?: ResponseBody, callback?: BodyResponseCallback): void;
/**
* Sniff an incoming HTTP response message for errors.
*
* @param {object} httpRespMessage - An incoming HTTP response message from `request`.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.resp - The original response object.
*/
parseHttpRespMessage(httpRespMessage: r.Response): ParsedHttpRespMessage;
/**
* Parse the response body from an HTTP request.
*
* @param {object} body - The response body.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.body - The original body value provided
* will try to be JSON.parse'd. If it's successful, the parsed value will
* be returned here, otherwise the original value and an error will be returned.
*/
parseHttpRespBody(body: ResponseBody): ParsedHttpResponseBody;
/**
* Take a Duplexify stream, fetch an authenticated connection header, and
* create an outgoing writable stream.
*
* @param {Duplexify} dup - Duplexify stream.
* @param {object} options - Configuration object.
* @param {module:common/connection} options.connection - A connection instance used to get a token with and send the request through.
* @param {object} options.metadata - Metadata to send at the head of the request.
* @param {object} options.request - Request object, in the format of a standard Node.js http.request() object.
* @param {string=} options.request.method - Default: "POST".
* @param {string=} options.request.qs.uploadType - Default: "multipart".
* @param {string=} options.streamContentType - Default: "application/octet-stream".
* @param {function} onComplete - Callback, executed after the writable Request stream has completed.
*/
makeWritableStream(dup: Duplexify, options: MakeWritableStreamOptions, onComplete?: Function): void;
/**
* Returns true if the API request should be retried, given the error that was
* given the first time the request was attempted. This is used for rate limit
* related errors as well as intermittent server errors.
*
* @param {error} err - The API error to check if it is appropriate to retry.
* @return {boolean} True if the API request should be retried, false otherwise.
*/
shouldRetryRequest(err?: ApiError): boolean;
/**
* Get a function for making authenticated requests.
*
* @param {object} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {object=} config.credentials - Credentials object.
* @param {boolean=} config.customEndpoint - If true, just return the provided request options. Default: false.
* @param {boolean=} config.useAuthWithCustomEndpoint - If true, will authenticate when using a custom endpoint. Default: false.
* @param {string=} config.email - Account email address, required for PEM/P12 usage.
* @param {number=} config.maxRetries - Maximum number of automatic retries attempted before returning the error. (default: 3)
* @param {string=} config.keyFile - Path to a .json, .pem, or .p12 keyfile.
* @param {array} config.scopes - Array of scopes required for the API.
*/
makeAuthenticatedRequestFactory(config: MakeAuthenticatedRequestFactoryConfig): MakeAuthenticatedRequest;
/**
* Make a request through the `retryRequest` module with built-in error
* handling and exponential back off.
*
* @param {object} reqOpts - Request options in the format `request` expects.
* @param {object=} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {number=} config.maxRetries - Maximum number of automatic retries
* attempted before returning the error. (default: 3)
* @param {object=} config.request - HTTP module for request calls.
* @param {function} callback - The callback function.
*/
makeRequest(reqOpts: DecorateRequestOptions, config: MakeRequestConfig, callback: BodyResponseCallback): void | Abortable;
/**
* Decorate the options about to be made in a request.
*
* @param {object} reqOpts - The options to be passed to `request`.
* @param {string} projectId - The project ID.
* @return {object} reqOpts - The decorated reqOpts.
*/
decorateRequest(reqOpts: DecorateRequestOptions, projectId: string): DecorateRequestOptions;
isCustomType(unknown: any, module: string): boolean;
/**
* Given two parameters, figure out if this is either:
* - Just a callback function
* - An options object, and then a callback function
* @param optionsOrCallback An options object or callback.
* @param cb A potentially undefined callback.
*/
maybeOptionsOrCallback<T = {}, C = (err?: Error) => void>(optionsOrCallback?: T | C, cb?: C): [T, C];
_getDefaultHeaders(gcclGcsCmd?: string): {
'User-Agent': string;
'x-goog-api-client': string;
};
}
declare const util: Util;
export { util };

View File

@@ -0,0 +1,669 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*!
* @module common/util
*/
import { replaceProjectIdToken, MissingProjectIdError, } from '@google-cloud/projectify';
import * as htmlEntities from 'html-entities';
import { GoogleAuth } from 'google-auth-library';
import retryRequest from 'retry-request';
import { Transform } from 'stream';
import { teenyRequest } from 'teeny-request';
import * as uuid from 'uuid';
import { DEFAULT_PROJECT_ID_TOKEN } from './service.js';
import { getModuleFormat, getRuntimeTrackingString, getUserAgentString, } from '../util.js';
import duplexify from 'duplexify';
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
import { getPackageJSON } from '../package-json-helper.cjs';
const packageJson = getPackageJSON();
/**
* A unique symbol for providing a `gccl-gcs-cmd` value
* for the `X-Goog-API-Client` header.
*
* E.g. the `V` in `X-Goog-API-Client: gccl-gcs-cmd/V`
**/
export const GCCL_GCS_CMD_KEY = Symbol.for('GCCL_GCS_CMD');
const requestDefaults = {
timeout: 60000,
gzip: true,
forever: true,
pool: {
maxSockets: Infinity,
},
};
/**
* Default behavior: Automatically retry retriable server errors.
*
* @const {boolean}
* @private
*/
const AUTO_RETRY_DEFAULT = true;
/**
* Default behavior: Only attempt to retry retriable errors 3 times.
*
* @const {number}
* @private
*/
const MAX_RETRY_DEFAULT = 3;
/**
* Custom error type for API errors.
*
* @param {object} errorBody - Error object.
*/
export class ApiError extends Error {
constructor(errorBodyOrMessage) {
super();
if (typeof errorBodyOrMessage !== 'object') {
this.message = errorBodyOrMessage || '';
return;
}
const errorBody = errorBodyOrMessage;
this.code = errorBody.code;
this.errors = errorBody.errors;
this.response = errorBody.response;
try {
this.errors = JSON.parse(this.response.body).error.errors;
}
catch (e) {
this.errors = errorBody.errors;
}
this.message = ApiError.createMultiErrorMessage(errorBody, this.errors);
Error.captureStackTrace(this);
}
/**
* Pieces together an error message by combining all unique error messages
* returned from a single GoogleError
*
* @private
*
* @param {GoogleErrorBody} err The original error.
* @param {GoogleInnerError[]} [errors] Inner errors, if any.
* @returns {string}
*/
static createMultiErrorMessage(err, errors) {
const messages = new Set();
if (err.message) {
messages.add(err.message);
}
if (errors && errors.length) {
errors.forEach(({ message }) => messages.add(message));
}
else if (err.response && err.response.body) {
messages.add(htmlEntities.decode(err.response.body.toString()));
}
else if (!err.message) {
messages.add('A failure occurred during this request.');
}
let messageArr = Array.from(messages);
if (messageArr.length > 1) {
messageArr = messageArr.map((message, i) => ` ${i + 1}. ${message}`);
messageArr.unshift('Multiple errors occurred during the request. Please see the `errors` array for complete details.\n');
messageArr.push('\n');
}
return messageArr.join('\n');
}
}
/**
* Custom error type for partial errors returned from the API.
*
* @param {object} b - Error object.
*/
export class PartialFailureError extends Error {
constructor(b) {
super();
const errorObject = b;
this.errors = errorObject.errors;
this.name = 'PartialFailureError';
this.response = errorObject.response;
this.message = ApiError.createMultiErrorMessage(errorObject, this.errors);
}
}
export class Util {
constructor() {
this.ApiError = ApiError;
this.PartialFailureError = PartialFailureError;
}
/**
* No op.
*
* @example
* function doSomething(callback) {
* callback = callback || noop;
* }
*/
noop() { }
/**
* Uniformly process an API response.
*
* @param {*} err - Error value.
* @param {*} resp - Response value.
* @param {*} body - Body value.
* @param {function} callback - The callback function.
*/
handleResp(err, resp, body, callback) {
callback = callback || util.noop;
const parsedResp = {
err: err || null,
...(resp && util.parseHttpRespMessage(resp)),
...(body && util.parseHttpRespBody(body)),
};
// Assign the parsed body to resp.body, even if { json: false } was passed
// as a request option.
// We assume that nobody uses the previously unparsed value of resp.body.
if (!parsedResp.err && resp && typeof parsedResp.body === 'object') {
parsedResp.resp.body = parsedResp.body;
}
if (parsedResp.err && resp) {
parsedResp.err.response = resp;
}
callback(parsedResp.err, parsedResp.body, parsedResp.resp);
}
/**
* Sniff an incoming HTTP response message for errors.
*
* @param {object} httpRespMessage - An incoming HTTP response message from `request`.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.resp - The original response object.
*/
parseHttpRespMessage(httpRespMessage) {
const parsedHttpRespMessage = {
resp: httpRespMessage,
};
if (httpRespMessage.statusCode < 200 || httpRespMessage.statusCode > 299) {
// Unknown error. Format according to ApiError standard.
parsedHttpRespMessage.err = new ApiError({
errors: new Array(),
code: httpRespMessage.statusCode,
message: httpRespMessage.statusMessage,
response: httpRespMessage,
});
}
return parsedHttpRespMessage;
}
/**
* Parse the response body from an HTTP request.
*
* @param {object} body - The response body.
* @return {object} parsedHttpRespMessage - The parsed response.
* @param {?error} parsedHttpRespMessage.err - An error detected.
* @param {object} parsedHttpRespMessage.body - The original body value provided
* will try to be JSON.parse'd. If it's successful, the parsed value will
* be returned here, otherwise the original value and an error will be returned.
*/
parseHttpRespBody(body) {
const parsedHttpRespBody = {
body,
};
if (typeof body === 'string') {
try {
parsedHttpRespBody.body = JSON.parse(body);
}
catch (err) {
parsedHttpRespBody.body = body;
}
}
if (parsedHttpRespBody.body && parsedHttpRespBody.body.error) {
// Error from JSON API.
parsedHttpRespBody.err = new ApiError(parsedHttpRespBody.body.error);
}
return parsedHttpRespBody;
}
/**
* Take a Duplexify stream, fetch an authenticated connection header, and
* create an outgoing writable stream.
*
* @param {Duplexify} dup - Duplexify stream.
* @param {object} options - Configuration object.
* @param {module:common/connection} options.connection - A connection instance used to get a token with and send the request through.
* @param {object} options.metadata - Metadata to send at the head of the request.
* @param {object} options.request - Request object, in the format of a standard Node.js http.request() object.
* @param {string=} options.request.method - Default: "POST".
* @param {string=} options.request.qs.uploadType - Default: "multipart".
* @param {string=} options.streamContentType - Default: "application/octet-stream".
* @param {function} onComplete - Callback, executed after the writable Request stream has completed.
*/
makeWritableStream(dup, options, onComplete) {
var _a;
onComplete = onComplete || util.noop;
const writeStream = new ProgressStream();
writeStream.on('progress', evt => dup.emit('progress', evt));
dup.setWritable(writeStream);
const defaultReqOpts = {
method: 'POST',
qs: {
uploadType: 'multipart',
},
timeout: 0,
maxRetries: 0,
};
const metadata = options.metadata || {};
const reqOpts = {
...defaultReqOpts,
...options.request,
qs: {
...defaultReqOpts.qs,
...(_a = options.request) === null || _a === void 0 ? void 0 : _a.qs,
},
multipart: [
{
'Content-Type': 'application/json',
body: JSON.stringify(metadata),
},
{
'Content-Type': metadata.contentType || 'application/octet-stream',
body: writeStream,
},
],
};
options.makeAuthenticatedRequest(reqOpts, {
onAuthenticated(err, authenticatedReqOpts) {
if (err) {
dup.destroy(err);
return;
}
requestDefaults.headers = util._getDefaultHeaders(reqOpts[GCCL_GCS_CMD_KEY]);
const request = teenyRequest.defaults(requestDefaults);
request(authenticatedReqOpts, (err, resp, body) => {
util.handleResp(err, resp, body, (err, data) => {
if (err) {
dup.destroy(err);
return;
}
dup.emit('response', resp);
onComplete(data);
});
});
},
});
}
/**
* Returns true if the API request should be retried, given the error that was
* given the first time the request was attempted. This is used for rate limit
* related errors as well as intermittent server errors.
*
* @param {error} err - The API error to check if it is appropriate to retry.
* @return {boolean} True if the API request should be retried, false otherwise.
*/
shouldRetryRequest(err) {
if (err) {
if ([408, 429, 500, 502, 503, 504].indexOf(err.code) !== -1) {
return true;
}
if (err.errors) {
for (const e of err.errors) {
const reason = e.reason;
if (reason === 'rateLimitExceeded') {
return true;
}
if (reason === 'userRateLimitExceeded') {
return true;
}
if (reason && reason.includes('EAI_AGAIN')) {
return true;
}
}
}
}
return false;
}
/**
* Get a function for making authenticated requests.
*
* @param {object} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {object=} config.credentials - Credentials object.
* @param {boolean=} config.customEndpoint - If true, just return the provided request options. Default: false.
* @param {boolean=} config.useAuthWithCustomEndpoint - If true, will authenticate when using a custom endpoint. Default: false.
* @param {string=} config.email - Account email address, required for PEM/P12 usage.
* @param {number=} config.maxRetries - Maximum number of automatic retries attempted before returning the error. (default: 3)
* @param {string=} config.keyFile - Path to a .json, .pem, or .p12 keyfile.
* @param {array} config.scopes - Array of scopes required for the API.
*/
makeAuthenticatedRequestFactory(config) {
const googleAutoAuthConfig = { ...config };
if (googleAutoAuthConfig.projectId === DEFAULT_PROJECT_ID_TOKEN) {
delete googleAutoAuthConfig.projectId;
}
let authClient;
if (googleAutoAuthConfig.authClient instanceof GoogleAuth) {
// Use an existing `GoogleAuth`
authClient = googleAutoAuthConfig.authClient;
}
else {
// Pass an `AuthClient` & `clientOptions` to `GoogleAuth`, if available
authClient = new GoogleAuth({
...googleAutoAuthConfig,
authClient: googleAutoAuthConfig.authClient,
clientOptions: googleAutoAuthConfig.clientOptions,
});
}
function makeAuthenticatedRequest(reqOpts, optionsOrCallback) {
let stream;
let projectId;
const reqConfig = { ...config };
let activeRequest_;
if (!optionsOrCallback) {
stream = duplexify();
reqConfig.stream = stream;
}
const options = typeof optionsOrCallback === 'object' ? optionsOrCallback : undefined;
const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : undefined;
async function setProjectId() {
projectId = await authClient.getProjectId();
}
const onAuthenticated = async (err, authenticatedReqOpts) => {
const authLibraryError = err;
const autoAuthFailed = err &&
typeof err.message === 'string' &&
err.message.indexOf('Could not load the default credentials') > -1;
if (autoAuthFailed) {
// Even though authentication failed, the API might not actually
// care.
authenticatedReqOpts = reqOpts;
}
if (!err || autoAuthFailed) {
try {
// Try with existing `projectId` value
authenticatedReqOpts = util.decorateRequest(authenticatedReqOpts, projectId);
err = null;
}
catch (e) {
if (e instanceof MissingProjectIdError) {
// A `projectId` was required, but we don't have one.
try {
// Attempt to get the `projectId`
await setProjectId();
authenticatedReqOpts = util.decorateRequest(authenticatedReqOpts, projectId);
err = null;
}
catch (e) {
// Re-use the "Could not load the default credentials error" if
// auto auth failed.
err = err || e;
}
}
else {
// Some other error unrelated to missing `projectId`
err = err || e;
}
}
}
if (err) {
if (stream) {
stream.destroy(err);
}
else {
const fn = options && options.onAuthenticated
? options.onAuthenticated
: callback;
fn(err);
}
return;
}
if (options && options.onAuthenticated) {
options.onAuthenticated(null, authenticatedReqOpts);
}
else {
activeRequest_ = util.makeRequest(authenticatedReqOpts, reqConfig, (apiResponseError, ...params) => {
if (apiResponseError &&
apiResponseError.code === 401 &&
authLibraryError) {
// Re-use the "Could not load the default credentials error" if
// the API request failed due to missing credentials.
apiResponseError = authLibraryError;
}
callback(apiResponseError, ...params);
});
}
};
const prepareRequest = async () => {
try {
const getProjectId = async () => {
if (config.projectId &&
config.projectId !== DEFAULT_PROJECT_ID_TOKEN) {
// The user provided a project ID. We don't need to check with the
// auth client, it could be incorrect.
return config.projectId;
}
if (config.projectIdRequired === false) {
// A projectId is not required. Return the default.
return DEFAULT_PROJECT_ID_TOKEN;
}
return setProjectId();
};
const authorizeRequest = async () => {
if (reqConfig.customEndpoint &&
!reqConfig.useAuthWithCustomEndpoint) {
// Using a custom API override. Do not use `google-auth-library` for
// authentication. (ex: connecting to a local Datastore server)
return reqOpts;
}
else {
return authClient.authorizeRequest(reqOpts);
}
};
const [_projectId, authorizedReqOpts] = await Promise.all([
getProjectId(),
authorizeRequest(),
]);
if (_projectId) {
projectId = _projectId;
}
return onAuthenticated(null, authorizedReqOpts);
}
catch (e) {
return onAuthenticated(e);
}
};
prepareRequest();
if (stream) {
return stream;
}
return {
abort() {
setImmediate(() => {
if (activeRequest_) {
activeRequest_.abort();
activeRequest_ = null;
}
});
},
};
}
const mar = makeAuthenticatedRequest;
mar.getCredentials = authClient.getCredentials.bind(authClient);
mar.authClient = authClient;
return mar;
}
/**
* Make a request through the `retryRequest` module with built-in error
* handling and exponential back off.
*
* @param {object} reqOpts - Request options in the format `request` expects.
* @param {object=} config - Configuration object.
* @param {boolean=} config.autoRetry - Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* (default: true)
* @param {number=} config.maxRetries - Maximum number of automatic retries
* attempted before returning the error. (default: 3)
* @param {object=} config.request - HTTP module for request calls.
* @param {function} callback - The callback function.
*/
makeRequest(reqOpts, config, callback) {
var _a, _b, _c, _d, _e;
let autoRetryValue = AUTO_RETRY_DEFAULT;
if (config.autoRetry !== undefined) {
autoRetryValue = config.autoRetry;
}
else if (((_a = config.retryOptions) === null || _a === void 0 ? void 0 : _a.autoRetry) !== undefined) {
autoRetryValue = config.retryOptions.autoRetry;
}
let maxRetryValue = MAX_RETRY_DEFAULT;
if (config.maxRetries !== undefined) {
maxRetryValue = config.maxRetries;
}
else if (((_b = config.retryOptions) === null || _b === void 0 ? void 0 : _b.maxRetries) !== undefined) {
maxRetryValue = config.retryOptions.maxRetries;
}
requestDefaults.headers = this._getDefaultHeaders(reqOpts[GCCL_GCS_CMD_KEY]);
const options = {
request: teenyRequest.defaults(requestDefaults),
retries: autoRetryValue !== false ? maxRetryValue : 0,
noResponseRetries: autoRetryValue !== false ? maxRetryValue : 0,
shouldRetryFn(httpRespMessage) {
var _a, _b;
const err = util.parseHttpRespMessage(httpRespMessage).err;
if ((_a = config.retryOptions) === null || _a === void 0 ? void 0 : _a.retryableErrorFn) {
return err && ((_b = config.retryOptions) === null || _b === void 0 ? void 0 : _b.retryableErrorFn(err));
}
return err && util.shouldRetryRequest(err);
},
maxRetryDelay: (_c = config.retryOptions) === null || _c === void 0 ? void 0 : _c.maxRetryDelay,
retryDelayMultiplier: (_d = config.retryOptions) === null || _d === void 0 ? void 0 : _d.retryDelayMultiplier,
totalTimeout: (_e = config.retryOptions) === null || _e === void 0 ? void 0 : _e.totalTimeout,
};
if (typeof reqOpts.maxRetries === 'number') {
options.retries = reqOpts.maxRetries;
options.noResponseRetries = reqOpts.maxRetries;
}
if (!config.stream) {
return retryRequest(reqOpts, options,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(err, response, body) => {
util.handleResp(err, response, body, callback);
});
}
const dup = config.stream;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
let requestStream;
const isGetRequest = (reqOpts.method || 'GET').toUpperCase() === 'GET';
if (isGetRequest) {
requestStream = retryRequest(reqOpts, options);
dup.setReadable(requestStream);
}
else {
// Streaming writable HTTP requests cannot be retried.
requestStream = options.request(reqOpts);
dup.setWritable(requestStream);
}
// Replay the Request events back to the stream.
requestStream
.on('error', dup.destroy.bind(dup))
.on('response', dup.emit.bind(dup, 'response'))
.on('complete', dup.emit.bind(dup, 'complete'));
dup.abort = requestStream.abort;
return dup;
}
/**
* Decorate the options about to be made in a request.
*
* @param {object} reqOpts - The options to be passed to `request`.
* @param {string} projectId - The project ID.
* @return {object} reqOpts - The decorated reqOpts.
*/
decorateRequest(reqOpts, projectId) {
delete reqOpts.autoPaginate;
delete reqOpts.autoPaginateVal;
delete reqOpts.objectMode;
if (reqOpts.qs !== null && typeof reqOpts.qs === 'object') {
delete reqOpts.qs.autoPaginate;
delete reqOpts.qs.autoPaginateVal;
reqOpts.qs = replaceProjectIdToken(reqOpts.qs, projectId);
}
if (Array.isArray(reqOpts.multipart)) {
reqOpts.multipart = reqOpts.multipart.map(part => {
return replaceProjectIdToken(part, projectId);
});
}
if (reqOpts.json !== null && typeof reqOpts.json === 'object') {
delete reqOpts.json.autoPaginate;
delete reqOpts.json.autoPaginateVal;
reqOpts.json = replaceProjectIdToken(reqOpts.json, projectId);
}
reqOpts.uri = replaceProjectIdToken(reqOpts.uri, projectId);
return reqOpts;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
isCustomType(unknown, module) {
function getConstructorName(obj) {
return obj.constructor && obj.constructor.name.toLowerCase();
}
const moduleNameParts = module.split('/');
const parentModuleName = moduleNameParts[0] && moduleNameParts[0].toLowerCase();
const subModuleName = moduleNameParts[1] && moduleNameParts[1].toLowerCase();
if (subModuleName && getConstructorName(unknown) !== subModuleName) {
return false;
}
let walkingModule = unknown;
// eslint-disable-next-line no-constant-condition
while (true) {
if (getConstructorName(walkingModule) === parentModuleName) {
return true;
}
walkingModule = walkingModule.parent;
if (!walkingModule) {
return false;
}
}
}
/**
* Given two parameters, figure out if this is either:
* - Just a callback function
* - An options object, and then a callback function
* @param optionsOrCallback An options object or callback.
* @param cb A potentially undefined callback.
*/
maybeOptionsOrCallback(optionsOrCallback, cb) {
return typeof optionsOrCallback === 'function'
? [{}, optionsOrCallback]
: [optionsOrCallback, cb];
}
_getDefaultHeaders(gcclGcsCmd) {
const headers = {
'User-Agent': getUserAgentString(),
'x-goog-api-client': `${getRuntimeTrackingString()} gccl/${packageJson.version}-${getModuleFormat()} gccl-invocation-id/${uuid.v4()}`,
};
if (gcclGcsCmd) {
headers['x-goog-api-client'] += ` gccl-gcs-cmd/${gcclGcsCmd}`;
}
return headers;
}
}
/**
* Basic Passthrough Stream that records the number of bytes read
* every time the cursor is moved.
*/
class ProgressStream extends Transform {
constructor() {
super(...arguments);
this.bytesRead = 0;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
_transform(chunk, encoding, callback) {
this.bytesRead += chunk.length;
this.emit('progress', { bytesWritten: this.bytesRead, contentLength: '*' });
this.push(chunk);
callback();
}
}
const util = new Util();
export { util };

View File

@@ -0,0 +1,106 @@
import { BaseMetadata, ServiceObject } from './nodejs-common/index.js';
import { ResponseBody } from './nodejs-common/util.js';
import { Bucket } from './bucket.js';
export interface DeleteNotificationOptions {
userProject?: string;
}
export interface GetNotificationMetadataOptions {
userProject?: string;
}
/**
* @typedef {array} GetNotificationMetadataResponse
* @property {object} 0 The notification metadata.
* @property {object} 1 The full API response.
*/
export type GetNotificationMetadataResponse = [ResponseBody, unknown];
/**
* @callback GetNotificationMetadataCallback
* @param {?Error} err Request error, if any.
* @param {object} files The notification metadata.
* @param {object} apiResponse The full API response.
*/
export interface GetNotificationMetadataCallback {
(err: Error | null, metadata?: ResponseBody, apiResponse?: unknown): void;
}
/**
* @typedef {array} GetNotificationResponse
* @property {Notification} 0 The {@link Notification}
* @property {object} 1 The full API response.
*/
export type GetNotificationResponse = [Notification, unknown];
export interface GetNotificationOptions {
/**
* Automatically create the object if it does not exist. Default: `false`.
*/
autoCreate?: boolean;
/**
* The ID of the project which will be billed for the request.
*/
userProject?: string;
}
/**
* @callback GetNotificationCallback
* @param {?Error} err Request error, if any.
* @param {Notification} notification The {@link Notification}.
* @param {object} apiResponse The full API response.
*/
export interface GetNotificationCallback {
(err: Error | null, notification?: Notification | null, apiResponse?: unknown): void;
}
/**
* @callback DeleteNotificationCallback
* @param {?Error} err Request error, if any.
* @param {object} apiResponse The full API response.
*/
export interface DeleteNotificationCallback {
(err: Error | null, apiResponse?: unknown): void;
}
export interface NotificationMetadata extends BaseMetadata {
custom_attributes?: {
[key: string]: string;
};
event_types?: string[];
object_name_prefix?: string;
payload_format?: 'JSON_API_V1' | 'NONE';
topic?: string;
}
/**
* The API-formatted resource description of the notification.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name Notification#metadata
* @type {object}
*/
/**
* A Notification object is created from your {@link Bucket} object using
* {@link Bucket#notification}. Use it to interact with Cloud Pub/Sub
* notifications.
*
* See {@link https://cloud.google.com/storage/docs/pubsub-notifications| Cloud Pub/Sub Notifications for Google Cloud Storage}
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket The bucket instance this notification is attached to.
* @param {string} id The ID of the notification.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const notification = myBucket.notification('1');
* ```
*/
declare class Notification extends ServiceObject<Notification, NotificationMetadata> {
constructor(bucket: Bucket, id: string);
}
/**
* Reference to the {@link Notification} class.
* @name module:@google-cloud/storage.Notification
* @see Notification
*/
export { Notification };

View File

@@ -0,0 +1,267 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { ServiceObject } from './nodejs-common/index.js';
import { promisifyAll } from '@google-cloud/promisify';
/**
* The API-formatted resource description of the notification.
*
* Note: This is not guaranteed to be up-to-date when accessed. To get the
* latest record, call the `getMetadata()` method.
*
* @name Notification#metadata
* @type {object}
*/
/**
* A Notification object is created from your {@link Bucket} object using
* {@link Bucket#notification}. Use it to interact with Cloud Pub/Sub
* notifications.
*
* See {@link https://cloud.google.com/storage/docs/pubsub-notifications| Cloud Pub/Sub Notifications for Google Cloud Storage}
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket The bucket instance this notification is attached to.
* @param {string} id The ID of the notification.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
*
* const notification = myBucket.notification('1');
* ```
*/
class Notification extends ServiceObject {
constructor(bucket, id) {
const requestQueryObject = {};
const methods = {
/**
* Creates a notification subscription for the bucket.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/insert| Notifications: insert}
* @method Notification#create
*
* @param {Topic|string} topic The Cloud PubSub topic to which this
* subscription publishes. If the project ID is omitted, the current
* project ID will be used.
*
* Acceptable formats are:
* - `projects/grape-spaceship-123/topics/my-topic`
*
* - `my-topic`
* @param {CreateNotificationRequest} [options] Metadata to set for
* the notification.
* @param {CreateNotificationCallback} [callback] Callback function.
* @returns {Promise<CreateNotificationResponse>}
* @throws {Error} If a valid topic is not provided.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.create(function(err, notification, apiResponse) {
* if (!err) {
* // The notification was created successfully.
* }
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.create().then(function(data) {
* const notification = data[0];
* const apiResponse = data[1];
* });
* ```
*/
create: true,
/**
* @typedef {array} DeleteNotificationResponse
* @property {object} 0 The full API response.
*/
/**
* Permanently deletes a notification subscription.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/delete| Notifications: delete API Documentation}
*
* @param {object} [options] Configuration options.
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {DeleteNotificationCallback} [callback] Callback function.
* @returns {Promise<DeleteNotificationResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.delete(function(err, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.delete().then(function(data) {
* const apiResponse = data[0];
* });
*
* ```
* @example <caption>include:samples/deleteNotification.js</caption>
* region_tag:storage_delete_bucket_notification
* Another example:
*/
delete: {
reqOpts: {
qs: requestQueryObject,
},
},
/**
* Get a notification and its metadata if it exists.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/get| Notifications: get API Documentation}
*
* @param {object} [options] Configuration options.
* See {@link Bucket#createNotification} for create options.
* @param {boolean} [options.autoCreate] Automatically create the object if
* it does not exist. Default: `false`.
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {GetNotificationCallback} [callback] Callback function.
* @return {Promise<GetNotificationCallback>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.get(function(err, notification, apiResponse) {
* // `notification.metadata` has been populated.
* });
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.get().then(function(data) {
* const notification = data[0];
* const apiResponse = data[1];
* });
* ```
*/
get: {
reqOpts: {
qs: requestQueryObject,
},
},
/**
* Get the notification's metadata.
*
* See {@link https://cloud.google.com/storage/docs/json_api/v1/notifications/get| Notifications: get API Documentation}
*
* @param {object} [options] Configuration options.
* @param {string} [options.userProject] The ID of the project which will be
* billed for the request.
* @param {GetNotificationMetadataCallback} [callback] Callback function.
* @returns {Promise<GetNotificationMetadataResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.getMetadata(function(err, metadata, apiResponse) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.getMetadata().then(function(data) {
* const metadata = data[0];
* const apiResponse = data[1];
* });
*
* ```
* @example <caption>include:samples/getMetadataNotifications.js</caption>
* region_tag:storage_print_pubsub_bucket_notification
* Another example:
*/
getMetadata: {
reqOpts: {
qs: requestQueryObject,
},
},
/**
* @typedef {array} NotificationExistsResponse
* @property {boolean} 0 Whether the notification exists or not.
*/
/**
* @callback NotificationExistsCallback
* @param {?Error} err Request error, if any.
* @param {boolean} exists Whether the notification exists or not.
*/
/**
* Check if the notification exists.
*
* @method Notification#exists
* @param {NotificationExistsCallback} [callback] Callback function.
* @returns {Promise<NotificationExistsResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const myBucket = storage.bucket('my-bucket');
* const notification = myBucket.notification('1');
*
* notification.exists(function(err, exists) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* notification.exists().then(function(data) {
* const exists = data[0];
* });
* ```
*/
exists: true,
};
super({
parent: bucket,
baseUrl: '/notificationConfigs',
id: id.toString(),
createMethod: bucket.createNotification.bind(bucket),
methods,
});
}
}
/*! Developer Documentation
*
* All async methods (except for streams) will return a Promise in the event
* that a callback is omitted.
*/
promisifyAll(Notification);
/**
* Reference to the {@link Notification} class.
* @name module:@google-cloud/storage.Notification
* @see Notification
*/
export { Notification };

View File

@@ -0,0 +1,21 @@
// Copyright 2023 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/* eslint-disable node/no-missing-require */
function getPackageJSON() {
return require('../../../package.json');
}
exports.getPackageJSON = getPackageJSON;

View File

@@ -0,0 +1,337 @@
import { GaxiosOptions, GaxiosPromise, GaxiosResponse } from 'gaxios';
import { GoogleAuthOptions } from 'google-auth-library';
import { Writable, WritableOptions } from 'stream';
import { RetryOptions, PreconditionOptions } from './storage.js';
import { GCCL_GCS_CMD_KEY } from './nodejs-common/util.js';
import { FileMetadata } from './file.js';
export declare const PROTOCOL_REGEX: RegExp;
export interface ErrorWithCode extends Error {
code: number;
status?: number | string;
}
export type CreateUriCallback = (err: Error | null, uri?: string) => void;
export interface Encryption {
key: {};
hash: {};
}
export type PredefinedAcl = 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
export interface QueryParameters extends PreconditionOptions {
contentEncoding?: string;
kmsKeyName?: string;
predefinedAcl?: PredefinedAcl;
projection?: 'full' | 'noAcl';
userProject?: string;
}
export interface UploadConfig extends Pick<WritableOptions, 'highWaterMark'> {
/**
* The API endpoint used for the request.
* Defaults to `storage.googleapis.com`.
*
* **Warning**:
* If this value does not match the current GCP universe an emulator context
* will be assumed and authentication will be bypassed.
*/
apiEndpoint?: string;
/**
* The name of the destination bucket.
*/
bucket: string;
/**
* The name of the destination file.
*/
file: string;
/**
* The GoogleAuthOptions passed to google-auth-library
*/
authConfig?: GoogleAuthOptions;
/**
* If you want to re-use an auth client from google-auto-auth, pass an
* instance here.
* Defaults to GoogleAuth and gets automatically overridden if an
* emulator context is detected.
*/
authClient?: {
request: <T>(opts: GaxiosOptions) => Promise<GaxiosResponse<T>> | GaxiosPromise<T>;
};
/**
* Create a separate request per chunk.
*
* This value is in bytes and should be a multiple of 256 KiB (2^18).
* We recommend using at least 8 MiB for the chunk size.
*
* @link https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
*/
chunkSize?: number;
/**
* For each API request we send, you may specify custom request options that
* we'll add onto the request. The request options follow the gaxios API:
* https://github.com/googleapis/gaxios#request-options.
*/
customRequestOptions?: GaxiosOptions;
/**
* This will cause the upload to fail if the current generation of the remote
* object does not match the one provided here.
*/
generation?: number;
/**
* Set to `true` if the upload is only a subset of the overall object to upload.
* This can be used when planning to continue the upload an object in another
* session.
*
* **Must be used with {@link UploadConfig.chunkSize} != `0`**.
*
* If this is a continuation of a previous upload, {@link UploadConfig.offset}
* should be set.
*
* @see {@link checkUploadStatus} for checking the status of an existing upload.
*/
isPartialUpload?: boolean;
/**
* A customer-supplied encryption key. See
* https://cloud.google.com/storage/docs/encryption#customer-supplied.
*/
key?: string | Buffer;
/**
* Resource name of the Cloud KMS key, of the form
* `projects/my-project/locations/global/keyRings/my-kr/cryptoKeys/my-key`,
* that will be used to encrypt the object. Overrides the object metadata's
* `kms_key_name` value, if any.
*/
kmsKeyName?: string;
/**
* Any metadata you wish to set on the object.
*/
metadata?: ConfigMetadata;
/**
* The starting byte in relation to the final uploaded object.
* **Must be used with {@link UploadConfig.uri}**.
*
* If resuming an interrupted stream, do not supply this argument unless you
* know the exact number of bytes the service has AND the provided stream's
* first byte is a continuation from that provided offset. If resuming an
* interrupted stream and this option has not been provided, we will treat
* the provided upload stream as the object to upload - where the first byte
* of the upload stream is the first byte of the object to upload; skipping
* any bytes that are already present on the server.
*
* @see {@link checkUploadStatus} for checking the status of an existing upload.
* @see {@link https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload#resume-upload.}
*/
offset?: number;
/**
* Set an Origin header when creating the resumable upload URI.
*/
origin?: string;
/**
* Specify query parameters that go along with the initial upload request. See
* https://cloud.google.com/storage/docs/json_api/v1/objects/insert#parameters
*/
params?: QueryParameters;
/**
* Apply a predefined set of access controls to the created file.
*/
predefinedAcl?: PredefinedAcl;
/**
* Make the uploaded file private. (Alias for config.predefinedAcl =
* 'private')
*/
private?: boolean;
/**
* Make the uploaded file public. (Alias for config.predefinedAcl =
* 'publicRead')
*/
public?: boolean;
/**
* The service domain for a given Cloud universe.
*/
universeDomain?: string;
/**
* If you already have a resumable URI from a previously-created resumable
* upload, just pass it in here and we'll use that.
*
* If resuming an interrupted stream and the {@link UploadConfig.offset}
* option has not been provided, we will treat the provided upload stream as
* the object to upload - where the first byte of the upload stream is the
* first byte of the object to upload; skipping any bytes that are already
* present on the server.
*
* @see {@link checkUploadStatus} for checking the status of an existing upload.
*/
uri?: string;
/**
* If the bucket being accessed has requesterPays functionality enabled, this
* can be set to control which project is billed for the access of this file.
*/
userProject?: string;
/**
* Configuration options for retrying retryable errors.
*/
retryOptions: RetryOptions;
[GCCL_GCS_CMD_KEY]?: string;
}
export interface ConfigMetadata {
[key: string]: any;
/**
* Set the length of the object being uploaded. If uploading a partial
* object, this is the overall size of the finalized object.
*/
contentLength?: number;
/**
* Set the content type of the incoming data.
*/
contentType?: string;
}
export interface GoogleInnerError {
reason?: string;
}
export interface ApiError extends Error {
code?: number;
errors?: GoogleInnerError[];
}
export interface CheckUploadStatusConfig {
/**
* Set to `false` to disable retries within this method.
*
* @defaultValue `true`
*/
retry?: boolean;
}
export declare class Upload extends Writable {
#private;
bucket: string;
file: string;
apiEndpoint: string;
baseURI: string;
authConfig?: {
scopes?: string[];
};
authClient: {
request: <T>(opts: GaxiosOptions) => Promise<GaxiosResponse<T>> | GaxiosPromise<T>;
};
cacheKey: string;
chunkSize?: number;
customRequestOptions: GaxiosOptions;
generation?: number;
key?: string | Buffer;
kmsKeyName?: string;
metadata: ConfigMetadata;
offset?: number;
origin?: string;
params: QueryParameters;
predefinedAcl?: PredefinedAcl;
private?: boolean;
public?: boolean;
uri?: string;
userProject?: string;
encryption?: Encryption;
uriProvidedManually: boolean;
numBytesWritten: number;
numRetries: number;
contentLength: number | '*';
retryOptions: RetryOptions;
timeOfFirstRequest: number;
isPartialUpload: boolean;
private currentInvocationId;
/**
* A cache of buffers written to this instance, ready for consuming
*/
private writeBuffers;
private numChunksReadInRequest;
/**
* An array of buffers used for caching the most recent upload chunk.
* We should not assume that the server received all bytes sent in the request.
* - https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
*/
private localWriteCache;
private localWriteCacheByteLength;
private upstreamEnded;
constructor(cfg: UploadConfig);
/**
* Prevent 'finish' event until the upload has succeeded.
*
* @param fireFinishEvent The finish callback
*/
_final(fireFinishEvent?: () => void): void;
/**
* Handles incoming data from upstream
*
* @param chunk The chunk to append to the buffer
* @param encoding The encoding of the chunk
* @param readCallback A callback for when the buffer has been read downstream
*/
_write(chunk: Buffer | string, encoding: BufferEncoding, readCallback?: () => void): void;
/**
* Prepends the local buffer to write buffer and resets it.
*
* @param keepLastBytes number of bytes to keep from the end of the local buffer.
*/
private prependLocalBufferToUpstream;
/**
* Retrieves data from upstream's buffer.
*
* @param limit The maximum amount to return from the buffer.
*/
private pullFromChunkBuffer;
/**
* A handler for determining if data is ready to be read from upstream.
*
* @returns If there will be more chunks to read in the future
*/
private waitForNextChunk;
/**
* Reads data from upstream up to the provided `limit`.
* Ends when the limit has reached or no data is expected to be pushed from upstream.
*
* @param limit The most amount of data this iterator should return. `Infinity` by default.
*/
private upstreamIterator;
createURI(): Promise<string>;
createURI(callback: CreateUriCallback): void;
protected createURIAsync(): Promise<string>;
private continueUploading;
startUploading(): Promise<void>;
private responseHandler;
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
checkUploadStatus(config?: CheckUploadStatusConfig): Promise<GaxiosResponse<FileMetadata | void>>;
private getAndSetOffset;
private makeRequest;
private makeRequestStream;
/**
* @return {bool} is the request good?
*/
private onResponse;
/**
* @param resp GaxiosResponse object from previous attempt
*/
private attemptDelayedRetry;
/**
* The amount of time to wait before retrying the request, in milliseconds.
* If negative, do not retry.
*
* @returns the amount of time to wait, in milliseconds.
*/
private getRetryDelay;
private sanitizeEndpoint;
/**
* Check if a given status code is 2xx
*
* @param status The status code to check
* @returns if the status is 2xx
*/
isSuccessfulResponse(status: number): boolean;
}
export declare function upload(cfg: UploadConfig): Upload;
export declare function createURI(cfg: UploadConfig): Promise<string>;
export declare function createURI(cfg: UploadConfig, callback: CreateUriCallback): void;
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
export declare function checkUploadStatus(cfg: UploadConfig & Required<Pick<UploadConfig, 'uri'>>): Promise<GaxiosResponse<void | FileMetadata>>;

View File

@@ -0,0 +1,863 @@
// Copyright 2022 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _Upload_instances, _Upload_gcclGcsCmd, _Upload_resetLocalBuffersCache, _Upload_addLocalBufferCache;
import AbortController from 'abort-controller';
import { createHash } from 'crypto';
import * as gaxios from 'gaxios';
import { DEFAULT_UNIVERSE, GoogleAuth, } from 'google-auth-library';
import { Readable, Writable } from 'stream';
import AsyncRetry from 'async-retry';
import * as uuid from 'uuid';
import { getRuntimeTrackingString, getModuleFormat, getUserAgentString, } from './util.js';
import { GCCL_GCS_CMD_KEY } from './nodejs-common/util.js';
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
import { getPackageJSON } from './package-json-helper.cjs';
const NOT_FOUND_STATUS_CODE = 404;
const RESUMABLE_INCOMPLETE_STATUS_CODE = 308;
const packageJson = getPackageJSON();
export const PROTOCOL_REGEX = /^(\w*):\/\//;
export class Upload extends Writable {
constructor(cfg) {
var _a;
super(cfg);
_Upload_instances.add(this);
this.numBytesWritten = 0;
this.numRetries = 0;
this.currentInvocationId = {
checkUploadStatus: uuid.v4(),
chunk: uuid.v4(),
uri: uuid.v4(),
};
/**
* A cache of buffers written to this instance, ready for consuming
*/
this.writeBuffers = [];
this.numChunksReadInRequest = 0;
/**
* An array of buffers used for caching the most recent upload chunk.
* We should not assume that the server received all bytes sent in the request.
* - https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
*/
this.localWriteCache = [];
this.localWriteCacheByteLength = 0;
this.upstreamEnded = false;
_Upload_gcclGcsCmd.set(this, void 0);
cfg = cfg || {};
if (!cfg.bucket || !cfg.file) {
throw new Error('A bucket and file name are required');
}
if (cfg.offset && !cfg.uri) {
throw new RangeError('Cannot provide an `offset` without providing a `uri`');
}
if (cfg.isPartialUpload && !cfg.chunkSize) {
throw new RangeError('Cannot set `isPartialUpload` without providing a `chunkSize`');
}
cfg.authConfig = cfg.authConfig || {};
cfg.authConfig.scopes = [
'https://www.googleapis.com/auth/devstorage.full_control',
];
this.authClient = cfg.authClient || new GoogleAuth(cfg.authConfig);
const universe = cfg.universeDomain || DEFAULT_UNIVERSE;
this.apiEndpoint = `https://storage.${universe}`;
if (cfg.apiEndpoint && cfg.apiEndpoint !== this.apiEndpoint) {
this.apiEndpoint = this.sanitizeEndpoint(cfg.apiEndpoint);
const hostname = new URL(this.apiEndpoint).hostname;
// check if it is a domain of a known universe
const isDomain = hostname === universe;
const isDefaultUniverseDomain = hostname === DEFAULT_UNIVERSE;
// check if it is a subdomain of a known universe
// by checking a last (universe's length + 1) of a hostname
const isSubDomainOfUniverse = hostname.slice(-(universe.length + 1)) === `.${universe}`;
const isSubDomainOfDefaultUniverse = hostname.slice(-(DEFAULT_UNIVERSE.length + 1)) ===
`.${DEFAULT_UNIVERSE}`;
if (!isDomain &&
!isDefaultUniverseDomain &&
!isSubDomainOfUniverse &&
!isSubDomainOfDefaultUniverse) {
// a custom, non-universe domain,
// use gaxios
this.authClient = gaxios;
}
}
this.baseURI = `${this.apiEndpoint}/upload/storage/v1/b`;
this.bucket = cfg.bucket;
const cacheKeyElements = [cfg.bucket, cfg.file];
if (typeof cfg.generation === 'number') {
cacheKeyElements.push(`${cfg.generation}`);
}
this.cacheKey = cacheKeyElements.join('/');
this.customRequestOptions = cfg.customRequestOptions || {};
this.file = cfg.file;
this.generation = cfg.generation;
this.kmsKeyName = cfg.kmsKeyName;
this.metadata = cfg.metadata || {};
this.offset = cfg.offset;
this.origin = cfg.origin;
this.params = cfg.params || {};
this.userProject = cfg.userProject;
this.chunkSize = cfg.chunkSize;
this.retryOptions = cfg.retryOptions;
this.isPartialUpload = (_a = cfg.isPartialUpload) !== null && _a !== void 0 ? _a : false;
if (cfg.key) {
if (typeof cfg.key === 'string') {
const base64Key = Buffer.from(cfg.key).toString('base64');
this.encryption = {
key: base64Key,
hash: createHash('sha256').update(cfg.key).digest('base64'),
};
}
else {
const base64Key = cfg.key.toString('base64');
this.encryption = {
key: base64Key,
hash: createHash('sha256').update(cfg.key).digest('base64'),
};
}
}
this.predefinedAcl = cfg.predefinedAcl;
if (cfg.private)
this.predefinedAcl = 'private';
if (cfg.public)
this.predefinedAcl = 'publicRead';
const autoRetry = cfg.retryOptions.autoRetry;
this.uriProvidedManually = !!cfg.uri;
this.uri = cfg.uri;
if (this.offset) {
// we're resuming an incomplete upload
this.numBytesWritten = this.offset;
}
this.numRetries = 0; // counter for number of retries currently executed
if (!autoRetry) {
cfg.retryOptions.maxRetries = 0;
}
this.timeOfFirstRequest = Date.now();
const contentLength = cfg.metadata
? Number(cfg.metadata.contentLength)
: NaN;
this.contentLength = isNaN(contentLength) ? '*' : contentLength;
__classPrivateFieldSet(this, _Upload_gcclGcsCmd, cfg[GCCL_GCS_CMD_KEY], "f");
this.once('writing', () => {
if (this.uri) {
this.continueUploading();
}
else {
this.createURI(err => {
if (err) {
return this.destroy(err);
}
this.startUploading();
return;
});
}
});
}
/**
* Prevent 'finish' event until the upload has succeeded.
*
* @param fireFinishEvent The finish callback
*/
_final(fireFinishEvent = () => { }) {
this.upstreamEnded = true;
this.once('uploadFinished', fireFinishEvent);
process.nextTick(() => {
this.emit('upstreamFinished');
// it's possible `_write` may not be called - namely for empty object uploads
this.emit('writing');
});
}
/**
* Handles incoming data from upstream
*
* @param chunk The chunk to append to the buffer
* @param encoding The encoding of the chunk
* @param readCallback A callback for when the buffer has been read downstream
*/
_write(chunk, encoding, readCallback = () => { }) {
// Backwards-compatible event
this.emit('writing');
this.writeBuffers.push(typeof chunk === 'string' ? Buffer.from(chunk, encoding) : chunk);
this.once('readFromChunkBuffer', readCallback);
process.nextTick(() => this.emit('wroteToChunkBuffer'));
}
/**
* Prepends the local buffer to write buffer and resets it.
*
* @param keepLastBytes number of bytes to keep from the end of the local buffer.
*/
prependLocalBufferToUpstream(keepLastBytes) {
// Typically, the upstream write buffers should be smaller than the local
// cache, so we can save time by setting the local cache as the new
// upstream write buffer array and appending the old array to it
let initialBuffers = [];
if (keepLastBytes) {
// we only want the last X bytes
let bytesKept = 0;
while (keepLastBytes > bytesKept) {
// load backwards because we want the last X bytes
// note: `localWriteCacheByteLength` is reset below
let buf = this.localWriteCache.pop();
if (!buf)
break;
bytesKept += buf.byteLength;
if (bytesKept > keepLastBytes) {
// we have gone over the amount desired, let's keep the last X bytes
// of this buffer
const diff = bytesKept - keepLastBytes;
buf = buf.subarray(diff);
bytesKept -= diff;
}
initialBuffers.unshift(buf);
}
}
else {
// we're keeping all of the local cache, simply use it as the initial buffer
initialBuffers = this.localWriteCache;
}
// Append the old upstream to the new
const append = this.writeBuffers;
this.writeBuffers = initialBuffers;
for (const buf of append) {
this.writeBuffers.push(buf);
}
// reset last buffers sent
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
}
/**
* Retrieves data from upstream's buffer.
*
* @param limit The maximum amount to return from the buffer.
*/
*pullFromChunkBuffer(limit) {
while (limit) {
const buf = this.writeBuffers.shift();
if (!buf)
break;
let bufToYield = buf;
if (buf.byteLength > limit) {
bufToYield = buf.subarray(0, limit);
this.writeBuffers.unshift(buf.subarray(limit));
limit = 0;
}
else {
limit -= buf.byteLength;
}
yield bufToYield;
// Notify upstream we've read from the buffer and we're able to consume
// more. It can also potentially send more data down as we're currently
// iterating.
this.emit('readFromChunkBuffer');
}
}
/**
* A handler for determining if data is ready to be read from upstream.
*
* @returns If there will be more chunks to read in the future
*/
async waitForNextChunk() {
const willBeMoreChunks = await new Promise(resolve => {
// There's data available - it should be digested
if (this.writeBuffers.length) {
return resolve(true);
}
// The upstream writable ended, we shouldn't expect any more data.
if (this.upstreamEnded) {
return resolve(false);
}
// Nothing immediate seems to be determined. We need to prepare some
// listeners to determine next steps...
const wroteToChunkBufferCallback = () => {
removeListeners();
return resolve(true);
};
const upstreamFinishedCallback = () => {
removeListeners();
// this should be the last chunk, if there's anything there
if (this.writeBuffers.length)
return resolve(true);
return resolve(false);
};
// Remove listeners when we're ready to callback.
const removeListeners = () => {
this.removeListener('wroteToChunkBuffer', wroteToChunkBufferCallback);
this.removeListener('upstreamFinished', upstreamFinishedCallback);
};
// If there's data recently written it should be digested
this.once('wroteToChunkBuffer', wroteToChunkBufferCallback);
// If the upstream finishes let's see if there's anything to grab
this.once('upstreamFinished', upstreamFinishedCallback);
});
return willBeMoreChunks;
}
/**
* Reads data from upstream up to the provided `limit`.
* Ends when the limit has reached or no data is expected to be pushed from upstream.
*
* @param limit The most amount of data this iterator should return. `Infinity` by default.
*/
async *upstreamIterator(limit = Infinity) {
// read from upstream chunk buffer
while (limit && (await this.waitForNextChunk())) {
// read until end or limit has been reached
for (const chunk of this.pullFromChunkBuffer(limit)) {
limit -= chunk.byteLength;
yield chunk;
}
}
}
createURI(callback) {
if (!callback) {
return this.createURIAsync();
}
this.createURIAsync().then(r => callback(null, r), callback);
}
async createURIAsync() {
const metadata = { ...this.metadata };
const headers = {};
// Delete content length and content type from metadata if they exist.
// These are headers and should not be sent as part of the metadata.
if (metadata.contentLength) {
headers['X-Upload-Content-Length'] = metadata.contentLength.toString();
delete metadata.contentLength;
}
if (metadata.contentType) {
headers['X-Upload-Content-Type'] = metadata.contentType;
delete metadata.contentType;
}
let googAPIClient = `${getRuntimeTrackingString()} gccl/${packageJson.version}-${getModuleFormat()} gccl-invocation-id/${this.currentInvocationId.uri}`;
if (__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")) {
googAPIClient += ` gccl-gcs-cmd/${__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")}`;
}
// Check if headers already exist before creating new ones
const reqOpts = {
method: 'POST',
url: [this.baseURI, this.bucket, 'o'].join('/'),
params: Object.assign({
name: this.file,
uploadType: 'resumable',
}, this.params),
data: metadata,
headers: {
'User-Agent': getUserAgentString(),
'x-goog-api-client': googAPIClient,
...headers,
},
};
if (metadata.contentLength) {
reqOpts.headers['X-Upload-Content-Length'] =
metadata.contentLength.toString();
}
if (metadata.contentType) {
reqOpts.headers['X-Upload-Content-Type'] = metadata.contentType;
}
if (typeof this.generation !== 'undefined') {
reqOpts.params.ifGenerationMatch = this.generation;
}
if (this.kmsKeyName) {
reqOpts.params.kmsKeyName = this.kmsKeyName;
}
if (this.predefinedAcl) {
reqOpts.params.predefinedAcl = this.predefinedAcl;
}
if (this.origin) {
reqOpts.headers.Origin = this.origin;
}
const uri = await AsyncRetry(async (bail) => {
var _a, _b, _c;
try {
const res = await this.makeRequest(reqOpts);
// We have successfully got a URI we can now create a new invocation id
this.currentInvocationId.uri = uuid.v4();
return res.headers.location;
}
catch (err) {
const e = err;
const apiError = {
code: (_a = e.response) === null || _a === void 0 ? void 0 : _a.status,
name: (_b = e.response) === null || _b === void 0 ? void 0 : _b.statusText,
message: (_c = e.response) === null || _c === void 0 ? void 0 : _c.statusText,
errors: [
{
reason: e.code,
},
],
};
if (this.retryOptions.maxRetries > 0 &&
this.retryOptions.retryableErrorFn(apiError)) {
throw e;
}
else {
return bail(e);
}
}
}, {
retries: this.retryOptions.maxRetries,
factor: this.retryOptions.retryDelayMultiplier,
maxTimeout: this.retryOptions.maxRetryDelay * 1000, //convert to milliseconds
maxRetryTime: this.retryOptions.totalTimeout * 1000, //convert to milliseconds
});
this.uri = uri;
this.offset = 0;
// emit the newly generated URI for future reuse, if necessary.
this.emit('uri', uri);
return uri;
}
async continueUploading() {
var _a;
(_a = this.offset) !== null && _a !== void 0 ? _a : (await this.getAndSetOffset());
return this.startUploading();
}
async startUploading() {
const multiChunkMode = !!this.chunkSize;
let responseReceived = false;
this.numChunksReadInRequest = 0;
if (!this.offset) {
this.offset = 0;
}
// Check if the offset (server) is too far behind the current stream
if (this.offset < this.numBytesWritten) {
const delta = this.numBytesWritten - this.offset;
const message = `The offset is lower than the number of bytes written. The server has ${this.offset} bytes and while ${this.numBytesWritten} bytes has been uploaded - thus ${delta} bytes are missing. Stopping as this could result in data loss. Initiate a new upload to continue.`;
this.emit('error', new RangeError(message));
return;
}
// Check if we should 'fast-forward' to the relevant data to upload
if (this.numBytesWritten < this.offset) {
// 'fast-forward' to the byte where we need to upload.
// only push data from the byte after the one we left off on
const fastForwardBytes = this.offset - this.numBytesWritten;
for await (const _chunk of this.upstreamIterator(fastForwardBytes)) {
_chunk; // discard the data up until the point we want
}
this.numBytesWritten = this.offset;
}
let expectedUploadSize = undefined;
// Set `expectedUploadSize` to `contentLength - this.numBytesWritten`, if available
if (typeof this.contentLength === 'number') {
expectedUploadSize = this.contentLength - this.numBytesWritten;
}
// `expectedUploadSize` should be no more than the `chunkSize`.
// It's possible this is the last chunk request for a multiple
// chunk upload, thus smaller than the chunk size.
if (this.chunkSize) {
expectedUploadSize = expectedUploadSize
? Math.min(this.chunkSize, expectedUploadSize)
: this.chunkSize;
}
// A queue for the upstream data
const upstreamQueue = this.upstreamIterator(expectedUploadSize);
// The primary read stream for this request. This stream retrieves no more
// than the exact requested amount from upstream.
const requestStream = new Readable({
read: async () => {
// Don't attempt to retrieve data upstream if we already have a response
if (responseReceived)
requestStream.push(null);
const result = await upstreamQueue.next();
if (result.value) {
this.numChunksReadInRequest++;
if (multiChunkMode) {
// save ever buffer used in the request in multi-chunk mode
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_addLocalBufferCache).call(this, result.value);
}
else {
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_addLocalBufferCache).call(this, result.value);
}
this.numBytesWritten += result.value.byteLength;
this.emit('progress', {
bytesWritten: this.numBytesWritten,
contentLength: this.contentLength,
});
requestStream.push(result.value);
}
if (result.done) {
requestStream.push(null);
}
},
});
let googAPIClient = `${getRuntimeTrackingString()} gccl/${packageJson.version}-${getModuleFormat()} gccl-invocation-id/${this.currentInvocationId.chunk}`;
if (__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")) {
googAPIClient += ` gccl-gcs-cmd/${__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")}`;
}
const headers = {
'User-Agent': getUserAgentString(),
'x-goog-api-client': googAPIClient,
};
// If using multiple chunk upload, set appropriate header
if (multiChunkMode) {
// We need to know how much data is available upstream to set the `Content-Range` header.
// https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
for await (const chunk of this.upstreamIterator(expectedUploadSize)) {
// This will conveniently track and keep the size of the buffers.
// We will reach either the expected upload size or the remainder of the stream.
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_addLocalBufferCache).call(this, chunk);
}
// This is the sum from the `#addLocalBufferCache` calls
const bytesToUpload = this.localWriteCacheByteLength;
// Important: we want to know if the upstream has ended and the queue is empty before
// unshifting data back into the queue. This way we will know if this is the last request or not.
const isLastChunkOfUpload = !(await this.waitForNextChunk());
// Important: put the data back in the queue for the actual upload
this.prependLocalBufferToUpstream();
let totalObjectSize = this.contentLength;
if (typeof this.contentLength !== 'number' &&
isLastChunkOfUpload &&
!this.isPartialUpload) {
// Let's let the server know this is the last chunk of the object since we didn't set it before.
totalObjectSize = bytesToUpload + this.numBytesWritten;
}
// `- 1` as the ending byte is inclusive in the request.
const endingByte = bytesToUpload + this.numBytesWritten - 1;
// `Content-Length` for multiple chunk uploads is the size of the chunk,
// not the overall object
headers['Content-Length'] = bytesToUpload;
headers['Content-Range'] =
`bytes ${this.offset}-${endingByte}/${totalObjectSize}`;
}
else {
headers['Content-Range'] = `bytes ${this.offset}-*/${this.contentLength}`;
}
const reqOpts = {
method: 'PUT',
url: this.uri,
headers,
body: requestStream,
};
try {
const resp = await this.makeRequestStream(reqOpts);
if (resp) {
responseReceived = true;
await this.responseHandler(resp);
}
}
catch (e) {
const err = e;
if (this.retryOptions.retryableErrorFn(err)) {
this.attemptDelayedRetry({
status: NaN,
data: err,
});
return;
}
this.destroy(err);
}
}
// Process the API response to look for errors that came in
// the response body.
async responseHandler(resp) {
if (resp.data.error) {
this.destroy(resp.data.error);
return;
}
// At this point we can safely create a new id for the chunk
this.currentInvocationId.chunk = uuid.v4();
const moreDataToUpload = await this.waitForNextChunk();
const shouldContinueWithNextMultiChunkRequest = this.chunkSize &&
resp.status === RESUMABLE_INCOMPLETE_STATUS_CODE &&
resp.headers.range &&
moreDataToUpload;
/**
* This is true when we're expecting to upload more data in a future request,
* yet the upstream for the upload session has been exhausted.
*/
const shouldContinueUploadInAnotherRequest = this.isPartialUpload &&
resp.status === RESUMABLE_INCOMPLETE_STATUS_CODE &&
!moreDataToUpload;
if (shouldContinueWithNextMultiChunkRequest) {
// Use the upper value in this header to determine where to start the next chunk.
// We should not assume that the server received all bytes sent in the request.
// https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
const range = resp.headers.range;
this.offset = Number(range.split('-')[1]) + 1;
// We should not assume that the server received all bytes sent in the request.
// - https://cloud.google.com/storage/docs/performing-resumable-uploads#chunked-upload
const missingBytes = this.numBytesWritten - this.offset;
if (missingBytes) {
// As multi-chunk uploads send one chunk per request and pulls one
// chunk into the pipeline, prepending the missing bytes back should
// be fine for the next request.
this.prependLocalBufferToUpstream(missingBytes);
this.numBytesWritten -= missingBytes;
}
else {
// No bytes missing - no need to keep the local cache
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
}
// continue uploading next chunk
this.continueUploading();
}
else if (!this.isSuccessfulResponse(resp.status) &&
!shouldContinueUploadInAnotherRequest) {
const err = new Error('Upload failed');
err.code = resp.status;
err.name = 'Upload failed';
if (resp === null || resp === void 0 ? void 0 : resp.data) {
err.errors = [resp === null || resp === void 0 ? void 0 : resp.data];
}
this.destroy(err);
}
else {
// no need to keep the cache
__classPrivateFieldGet(this, _Upload_instances, "m", _Upload_resetLocalBuffersCache).call(this);
if (resp && resp.data) {
resp.data.size = Number(resp.data.size);
}
this.emit('metadata', resp.data);
// Allow the object (Upload) to continue naturally so the user's
// "finish" event fires.
this.emit('uploadFinished');
}
}
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
async checkUploadStatus(config = {}) {
let googAPIClient = `${getRuntimeTrackingString()} gccl/${packageJson.version}-${getModuleFormat()} gccl-invocation-id/${this.currentInvocationId.checkUploadStatus}`;
if (__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")) {
googAPIClient += ` gccl-gcs-cmd/${__classPrivateFieldGet(this, _Upload_gcclGcsCmd, "f")}`;
}
const opts = {
method: 'PUT',
url: this.uri,
headers: {
'Content-Length': 0,
'Content-Range': 'bytes */*',
'User-Agent': getUserAgentString(),
'x-goog-api-client': googAPIClient,
},
};
try {
const resp = await this.makeRequest(opts);
// Successfully got the offset we can now create a new offset invocation id
this.currentInvocationId.checkUploadStatus = uuid.v4();
return resp;
}
catch (e) {
if (config.retry === false ||
!(e instanceof Error) ||
!this.retryOptions.retryableErrorFn(e)) {
throw e;
}
const retryDelay = this.getRetryDelay();
if (retryDelay <= 0) {
throw e;
}
await new Promise(res => setTimeout(res, retryDelay));
return this.checkUploadStatus(config);
}
}
async getAndSetOffset() {
try {
// we want to handle retries in this method.
const resp = await this.checkUploadStatus({ retry: false });
if (resp.status === RESUMABLE_INCOMPLETE_STATUS_CODE) {
if (typeof resp.headers.range === 'string') {
this.offset = Number(resp.headers.range.split('-')[1]) + 1;
return;
}
}
this.offset = 0;
}
catch (e) {
const err = e;
if (this.retryOptions.retryableErrorFn(err)) {
this.attemptDelayedRetry({
status: NaN,
data: err,
});
return;
}
this.destroy(err);
}
}
async makeRequest(reqOpts) {
if (this.encryption) {
reqOpts.headers = reqOpts.headers || {};
reqOpts.headers['x-goog-encryption-algorithm'] = 'AES256';
reqOpts.headers['x-goog-encryption-key'] = this.encryption.key.toString();
reqOpts.headers['x-goog-encryption-key-sha256'] =
this.encryption.hash.toString();
}
if (this.userProject) {
reqOpts.params = reqOpts.params || {};
reqOpts.params.userProject = this.userProject;
}
// Let gaxios know we will handle a 308 error code ourselves.
reqOpts.validateStatus = (status) => {
return (this.isSuccessfulResponse(status) ||
status === RESUMABLE_INCOMPLETE_STATUS_CODE);
};
const combinedReqOpts = {
...this.customRequestOptions,
...reqOpts,
headers: {
...this.customRequestOptions.headers,
...reqOpts.headers,
},
};
const res = await this.authClient.request(combinedReqOpts);
if (res.data && res.data.error) {
throw res.data.error;
}
return res;
}
async makeRequestStream(reqOpts) {
const controller = new AbortController();
const errorCallback = () => controller.abort();
this.once('error', errorCallback);
if (this.userProject) {
reqOpts.params = reqOpts.params || {};
reqOpts.params.userProject = this.userProject;
}
reqOpts.signal = controller.signal;
reqOpts.validateStatus = () => true;
const combinedReqOpts = {
...this.customRequestOptions,
...reqOpts,
headers: {
...this.customRequestOptions.headers,
...reqOpts.headers,
},
};
const res = await this.authClient.request(combinedReqOpts);
const successfulRequest = this.onResponse(res);
this.removeListener('error', errorCallback);
return successfulRequest ? res : null;
}
/**
* @return {bool} is the request good?
*/
onResponse(resp) {
if (resp.status !== 200 &&
this.retryOptions.retryableErrorFn({
code: resp.status,
message: resp.statusText,
name: resp.statusText,
})) {
this.attemptDelayedRetry(resp);
return false;
}
this.emit('response', resp);
return true;
}
/**
* @param resp GaxiosResponse object from previous attempt
*/
attemptDelayedRetry(resp) {
if (this.numRetries < this.retryOptions.maxRetries) {
if (resp.status === NOT_FOUND_STATUS_CODE &&
this.numChunksReadInRequest === 0) {
this.startUploading();
}
else {
const retryDelay = this.getRetryDelay();
if (retryDelay <= 0) {
this.destroy(new Error(`Retry total time limit exceeded - ${JSON.stringify(resp.data)}`));
return;
}
// Unshift the local cache back in case it's needed for the next request.
this.numBytesWritten -= this.localWriteCacheByteLength;
this.prependLocalBufferToUpstream();
// We don't know how much data has been received by the server.
// `continueUploading` will recheck the offset via `getAndSetOffset`.
// If `offset` < `numberBytesReceived` then we will raise a RangeError
// as we've streamed too much data that has been missed - this should
// not be the case for multi-chunk uploads as `lastChunkSent` is the
// body of the entire request.
this.offset = undefined;
setTimeout(this.continueUploading.bind(this), retryDelay);
}
this.numRetries++;
}
else {
this.destroy(new Error(`Retry limit exceeded - ${JSON.stringify(resp.data)}`));
}
}
/**
* The amount of time to wait before retrying the request, in milliseconds.
* If negative, do not retry.
*
* @returns the amount of time to wait, in milliseconds.
*/
getRetryDelay() {
const randomMs = Math.round(Math.random() * 1000);
const waitTime = Math.pow(this.retryOptions.retryDelayMultiplier, this.numRetries) *
1000 +
randomMs;
const maxAllowableDelayMs = this.retryOptions.totalTimeout * 1000 -
(Date.now() - this.timeOfFirstRequest);
const maxRetryDelayMs = this.retryOptions.maxRetryDelay * 1000;
return Math.min(waitTime, maxRetryDelayMs, maxAllowableDelayMs);
}
/*
* Prepare user-defined API endpoint for compatibility with our API.
*/
sanitizeEndpoint(url) {
if (!PROTOCOL_REGEX.test(url)) {
url = `https://${url}`;
}
return url.replace(/\/+$/, ''); // Remove trailing slashes
}
/**
* Check if a given status code is 2xx
*
* @param status The status code to check
* @returns if the status is 2xx
*/
isSuccessfulResponse(status) {
return status >= 200 && status < 300;
}
}
_Upload_gcclGcsCmd = new WeakMap(), _Upload_instances = new WeakSet(), _Upload_resetLocalBuffersCache = function _Upload_resetLocalBuffersCache() {
this.localWriteCache = [];
this.localWriteCacheByteLength = 0;
}, _Upload_addLocalBufferCache = function _Upload_addLocalBufferCache(buf) {
this.localWriteCache.push(buf);
this.localWriteCacheByteLength += buf.byteLength;
};
export function upload(cfg) {
return new Upload(cfg);
}
export function createURI(cfg, callback) {
const up = new Upload(cfg);
if (!callback) {
return up.createURI();
}
up.createURI().then(r => callback(null, r), callback);
}
/**
* Check the status of an existing resumable upload.
*
* @param cfg A configuration to use. `uri` is required.
* @returns the current upload status
*/
export function checkUploadStatus(cfg) {
const up = new Upload(cfg);
return up.checkUploadStatus();
}

View File

@@ -0,0 +1,146 @@
import * as http from 'http';
import { Storage } from './storage.js';
import { GoogleAuth } from 'google-auth-library';
type GoogleAuthLike = Pick<GoogleAuth, 'getCredentials' | 'sign'>;
/**
* @deprecated Use {@link GoogleAuth} instead
*/
export interface AuthClient {
sign(blobToSign: string): Promise<string>;
getCredentials(): Promise<{
client_email?: string;
}>;
}
export interface BucketI {
name: string;
}
export interface FileI {
name: string;
}
export interface Query {
[key: string]: string;
}
export interface GetSignedUrlConfigInternal {
expiration: number;
accessibleAt?: Date;
method: string;
extensionHeaders?: http.OutgoingHttpHeaders;
queryParams?: Query;
cname?: string;
contentMd5?: string;
contentType?: string;
bucket: string;
file?: string;
/**
* The host for the generated signed URL
*
* @example
* 'https://localhost:8080/'
*/
host?: string | URL;
/**
* An endpoint for generating the signed URL
*
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string | URL;
}
export interface SignerGetSignedUrlConfig {
method: 'GET' | 'PUT' | 'DELETE' | 'POST';
expires: string | number | Date;
accessibleAt?: string | number | Date;
virtualHostedStyle?: boolean;
version?: 'v2' | 'v4';
cname?: string;
extensionHeaders?: http.OutgoingHttpHeaders;
queryParams?: Query;
contentMd5?: string;
contentType?: string;
/**
* The host for the generated signed URL
*
* @example
* 'https://localhost:8080/'
*/
host?: string | URL;
/**
* An endpoint for generating the signed URL
*
* @example
* 'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/'
*/
signingEndpoint?: string | URL;
}
export type SignerGetSignedUrlResponse = string;
export type GetSignedUrlResponse = [SignerGetSignedUrlResponse];
export interface GetSignedUrlCallback {
(err: Error | null, url?: string): void;
}
export declare enum SignerExceptionMessages {
ACCESSIBLE_DATE_INVALID = "The accessible at date provided was invalid.",
EXPIRATION_BEFORE_ACCESSIBLE_DATE = "An expiration date cannot be before accessible date.",
X_GOOG_CONTENT_SHA256 = "The header X-Goog-Content-SHA256 must be a hexadecimal string."
}
/**
* @const {string}
* @deprecated - unused
*/
export declare const PATH_STYLED_HOST = "https://storage.googleapis.com";
export declare class URLSigner {
private auth;
private bucket;
private file?;
/**
* A {@link Storage} object.
*
* @privateRemarks
*
* Technically this is a required field, however it would be a breaking change to
* move it before optional properties. In the next major we should refactor the
* constructor of this class to only accept a config object.
*/
private storage;
constructor(auth: AuthClient | GoogleAuthLike, bucket: BucketI, file?: FileI | undefined,
/**
* A {@link Storage} object.
*
* @privateRemarks
*
* Technically this is a required field, however it would be a breaking change to
* move it before optional properties. In the next major we should refactor the
* constructor of this class to only accept a config object.
*/
storage?: Storage);
getSignedUrl(cfg: SignerGetSignedUrlConfig): Promise<SignerGetSignedUrlResponse>;
private getSignedUrlV2;
private getSignedUrlV4;
/**
* Create canonical headers for signing v4 url.
*
* The canonical headers for v4-signing a request demands header names are
* first lowercased, followed by sorting the header names.
* Then, construct the canonical headers part of the request:
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
* ..
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
*
* @param headers
* @private
*/
getCanonicalHeaders(headers: http.OutgoingHttpHeaders): string;
getCanonicalRequest(method: string, path: string, query: string, headers: string, signedHeaders: string, contentSha256?: string): string;
getCanonicalQueryParams(query: Query): string;
getResourcePath(cname: boolean, bucket: string, file?: string): string;
parseExpires(expires: string | number | Date, current?: Date): number;
parseAccessibleAt(accessibleAt?: string | number | Date): number;
}
/**
* Custom error type for errors related to getting signed errors and policies.
*
* @private
*/
export declare class SigningError extends Error {
name: string;
}
export {};

View File

@@ -0,0 +1,299 @@
// Copyright 2020 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import * as crypto from 'crypto';
import * as url from 'url';
import { ExceptionMessages, Storage } from './storage.js';
import { encodeURI, qsStringify, objectEntries, formatAsUTCISO } from './util.js';
export var SignerExceptionMessages;
(function (SignerExceptionMessages) {
SignerExceptionMessages["ACCESSIBLE_DATE_INVALID"] = "The accessible at date provided was invalid.";
SignerExceptionMessages["EXPIRATION_BEFORE_ACCESSIBLE_DATE"] = "An expiration date cannot be before accessible date.";
SignerExceptionMessages["X_GOOG_CONTENT_SHA256"] = "The header X-Goog-Content-SHA256 must be a hexadecimal string.";
})(SignerExceptionMessages || (SignerExceptionMessages = {}));
/*
* Default signing version for getSignedUrl is 'v2'.
*/
const DEFAULT_SIGNING_VERSION = 'v2';
const SEVEN_DAYS = 7 * 24 * 60 * 60;
/**
* @const {string}
* @deprecated - unused
*/
export const PATH_STYLED_HOST = 'https://storage.googleapis.com';
export class URLSigner {
constructor(auth, bucket, file,
/**
* A {@link Storage} object.
*
* @privateRemarks
*
* Technically this is a required field, however it would be a breaking change to
* move it before optional properties. In the next major we should refactor the
* constructor of this class to only accept a config object.
*/
storage = new Storage()) {
this.auth = auth;
this.bucket = bucket;
this.file = file;
this.storage = storage;
}
getSignedUrl(cfg) {
const expiresInSeconds = this.parseExpires(cfg.expires);
const method = cfg.method;
const accessibleAtInSeconds = this.parseAccessibleAt(cfg.accessibleAt);
if (expiresInSeconds < accessibleAtInSeconds) {
throw new Error(SignerExceptionMessages.EXPIRATION_BEFORE_ACCESSIBLE_DATE);
}
let customHost;
// Default style is `path`.
const isVirtualHostedStyle = cfg.virtualHostedStyle || false;
if (cfg.cname) {
customHost = cfg.cname;
}
else if (isVirtualHostedStyle) {
customHost = `https://${this.bucket.name}.storage.${this.storage.universeDomain}`;
}
const secondsToMilliseconds = 1000;
const config = Object.assign({}, cfg, {
method,
expiration: expiresInSeconds,
accessibleAt: new Date(secondsToMilliseconds * accessibleAtInSeconds),
bucket: this.bucket.name,
file: this.file ? encodeURI(this.file.name, false) : undefined,
});
if (customHost) {
config.cname = customHost;
}
const version = cfg.version || DEFAULT_SIGNING_VERSION;
let promise;
if (version === 'v2') {
promise = this.getSignedUrlV2(config);
}
else if (version === 'v4') {
promise = this.getSignedUrlV4(config);
}
else {
throw new Error(`Invalid signed URL version: ${version}. Supported versions are 'v2' and 'v4'.`);
}
return promise.then(query => {
var _a;
query = Object.assign(query, cfg.queryParams);
const signedUrl = new url.URL(((_a = cfg.host) === null || _a === void 0 ? void 0 : _a.toString()) || config.cname || this.storage.apiEndpoint);
signedUrl.pathname = this.getResourcePath(!!config.cname, this.bucket.name, config.file);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
signedUrl.search = qsStringify(query);
return signedUrl.href;
});
}
getSignedUrlV2(config) {
const canonicalHeadersString = this.getCanonicalHeaders(config.extensionHeaders || {});
const resourcePath = this.getResourcePath(false, config.bucket, config.file);
const blobToSign = [
config.method,
config.contentMd5 || '',
config.contentType || '',
config.expiration,
canonicalHeadersString + resourcePath,
].join('\n');
const sign = async () => {
var _a;
const auth = this.auth;
try {
const signature = await auth.sign(blobToSign, (_a = config.signingEndpoint) === null || _a === void 0 ? void 0 : _a.toString());
const credentials = await auth.getCredentials();
return {
GoogleAccessId: credentials.client_email,
Expires: config.expiration,
Signature: signature,
};
}
catch (err) {
const error = err;
const signingErr = new SigningError(error.message);
signingErr.stack = error.stack;
throw signingErr;
}
};
return sign();
}
getSignedUrlV4(config) {
var _a;
config.accessibleAt = config.accessibleAt
? config.accessibleAt
: new Date();
const millisecondsToSeconds = 1.0 / 1000.0;
const expiresPeriodInSeconds = config.expiration - config.accessibleAt.valueOf() * millisecondsToSeconds;
// v4 limit expiration to be 7 days maximum
if (expiresPeriodInSeconds > SEVEN_DAYS) {
throw new Error(`Max allowed expiration is seven days (${SEVEN_DAYS} seconds).`);
}
const extensionHeaders = Object.assign({}, config.extensionHeaders);
const fqdn = new url.URL(((_a = config.host) === null || _a === void 0 ? void 0 : _a.toString()) || config.cname || this.storage.apiEndpoint);
extensionHeaders.host = fqdn.hostname;
if (config.contentMd5) {
extensionHeaders['content-md5'] = config.contentMd5;
}
if (config.contentType) {
extensionHeaders['content-type'] = config.contentType;
}
let contentSha256;
const sha256Header = extensionHeaders['x-goog-content-sha256'];
if (sha256Header) {
if (typeof sha256Header !== 'string' ||
!/[A-Fa-f0-9]{40}/.test(sha256Header)) {
throw new Error(SignerExceptionMessages.X_GOOG_CONTENT_SHA256);
}
contentSha256 = sha256Header;
}
const signedHeaders = Object.keys(extensionHeaders)
.map(header => header.toLowerCase())
.sort()
.join(';');
const extensionHeadersString = this.getCanonicalHeaders(extensionHeaders);
const datestamp = formatAsUTCISO(config.accessibleAt);
const credentialScope = `${datestamp}/auto/storage/goog4_request`;
const sign = async () => {
var _a;
const credentials = await this.auth.getCredentials();
const credential = `${credentials.client_email}/${credentialScope}`;
const dateISO = formatAsUTCISO(config.accessibleAt ? config.accessibleAt : new Date(), true);
const queryParams = {
'X-Goog-Algorithm': 'GOOG4-RSA-SHA256',
'X-Goog-Credential': credential,
'X-Goog-Date': dateISO,
'X-Goog-Expires': expiresPeriodInSeconds.toString(10),
'X-Goog-SignedHeaders': signedHeaders,
...(config.queryParams || {}),
};
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const canonicalQueryParams = this.getCanonicalQueryParams(queryParams);
const canonicalRequest = this.getCanonicalRequest(config.method, this.getResourcePath(!!config.cname, config.bucket, config.file), canonicalQueryParams, extensionHeadersString, signedHeaders, contentSha256);
const hash = crypto
.createHash('sha256')
.update(canonicalRequest)
.digest('hex');
const blobToSign = [
'GOOG4-RSA-SHA256',
dateISO,
credentialScope,
hash,
].join('\n');
try {
const signature = await this.auth.sign(blobToSign, (_a = config.signingEndpoint) === null || _a === void 0 ? void 0 : _a.toString());
const signatureHex = Buffer.from(signature, 'base64').toString('hex');
const signedQuery = Object.assign({}, queryParams, {
'X-Goog-Signature': signatureHex,
});
return signedQuery;
}
catch (err) {
const error = err;
const signingErr = new SigningError(error.message);
signingErr.stack = error.stack;
throw signingErr;
}
};
return sign();
}
/**
* Create canonical headers for signing v4 url.
*
* The canonical headers for v4-signing a request demands header names are
* first lowercased, followed by sorting the header names.
* Then, construct the canonical headers part of the request:
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
* ..
* <lowercasedHeaderName> + ":" + Trim(<value>) + "\n"
*
* @param headers
* @private
*/
getCanonicalHeaders(headers) {
// Sort headers by their lowercased names
const sortedHeaders = objectEntries(headers)
// Convert header names to lowercase
.map(([headerName, value]) => [
headerName.toLowerCase(),
value,
])
.sort((a, b) => a[0].localeCompare(b[0]));
return sortedHeaders
.filter(([, value]) => value !== undefined)
.map(([headerName, value]) => {
// - Convert Array (multi-valued header) into string, delimited by
// ',' (no space).
// - Trim leading and trailing spaces.
// - Convert sequential (2+) spaces into a single space
const canonicalValue = `${value}`.trim().replace(/\s{2,}/g, ' ');
return `${headerName}:${canonicalValue}\n`;
})
.join('');
}
getCanonicalRequest(method, path, query, headers, signedHeaders, contentSha256) {
return [
method,
path,
query,
headers,
signedHeaders,
contentSha256 || 'UNSIGNED-PAYLOAD',
].join('\n');
}
getCanonicalQueryParams(query) {
return objectEntries(query)
.map(([key, value]) => [encodeURI(key, true), encodeURI(value, true)])
.sort((a, b) => (a[0] < b[0] ? -1 : 1))
.map(([key, value]) => `${key}=${value}`)
.join('&');
}
getResourcePath(cname, bucket, file) {
if (cname) {
return '/' + (file || '');
}
else if (file) {
return `/${bucket}/${file}`;
}
else {
return `/${bucket}`;
}
}
parseExpires(expires, current = new Date()) {
const expiresInMSeconds = new Date(expires).valueOf();
if (isNaN(expiresInMSeconds)) {
throw new Error(ExceptionMessages.EXPIRATION_DATE_INVALID);
}
if (expiresInMSeconds < current.valueOf()) {
throw new Error(ExceptionMessages.EXPIRATION_DATE_PAST);
}
return Math.floor(expiresInMSeconds / 1000); // The API expects seconds.
}
parseAccessibleAt(accessibleAt) {
const accessibleAtInMSeconds = new Date(accessibleAt || new Date()).valueOf();
if (isNaN(accessibleAtInMSeconds)) {
throw new Error(SignerExceptionMessages.ACCESSIBLE_DATE_INVALID);
}
return Math.floor(accessibleAtInMSeconds / 1000); // The API expects seconds.
}
}
/**
* Custom error type for errors related to getting signed errors and policies.
*
* @private
*/
export class SigningError extends Error {
constructor() {
super(...arguments);
this.name = 'SigningError';
}
}

View File

@@ -0,0 +1,729 @@
import { ApiError, Service, ServiceOptions } from './nodejs-common/index.js';
import { Readable } from 'stream';
import { Bucket, BucketMetadata } from './bucket.js';
import { Channel } from './channel.js';
import { File } from './file.js';
import { HmacKey, HmacKeyMetadata, HmacKeyOptions } from './hmacKey.js';
import { CRC32CValidatorGenerator } from './crc32c.js';
export interface GetServiceAccountOptions {
userProject?: string;
projectIdentifier?: string;
}
export interface ServiceAccount {
emailAddress?: string;
}
export type GetServiceAccountResponse = [ServiceAccount, unknown];
export interface GetServiceAccountCallback {
(err: Error | null, serviceAccount?: ServiceAccount, apiResponse?: unknown): void;
}
export interface CreateBucketQuery {
enableObjectRetention: boolean;
predefinedAcl?: 'authenticatedRead' | 'private' | 'projectPrivate' | 'publicRead' | 'publicReadWrite';
predefinedDefaultObjectAcl?: 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
project: string;
projection?: 'full' | 'noAcl';
userProject: string;
}
export declare enum IdempotencyStrategy {
RetryAlways = 0,
RetryConditional = 1,
RetryNever = 2
}
export interface RetryOptions {
retryDelayMultiplier?: number;
totalTimeout?: number;
maxRetryDelay?: number;
autoRetry?: boolean;
maxRetries?: number;
retryableErrorFn?: (err: ApiError) => boolean;
idempotencyStrategy?: IdempotencyStrategy;
}
export interface PreconditionOptions {
ifGenerationMatch?: number | string;
ifGenerationNotMatch?: number | string;
ifMetagenerationMatch?: number | string;
ifMetagenerationNotMatch?: number | string;
}
export interface StorageOptions extends ServiceOptions {
/**
* The API endpoint of the service used to make requests.
* Defaults to `storage.googleapis.com`.
*/
apiEndpoint?: string;
crc32cGenerator?: CRC32CValidatorGenerator;
retryOptions?: RetryOptions;
}
export interface BucketOptions {
crc32cGenerator?: CRC32CValidatorGenerator;
kmsKeyName?: string;
preconditionOpts?: PreconditionOptions;
userProject?: string;
generation?: number;
softDeleted?: boolean;
}
export interface Cors {
maxAgeSeconds?: number;
method?: string[];
origin?: string[];
responseHeader?: string[];
}
interface Versioning {
enabled: boolean;
}
/**
* Custom placement configuration.
* Initially used for dual-region buckets.
**/
export interface CustomPlacementConfig {
dataLocations?: string[];
}
export interface AutoclassConfig {
enabled?: boolean;
terminalStorageClass?: 'NEARLINE' | 'ARCHIVE';
}
export interface CreateBucketRequest extends BucketMetadata {
archive?: boolean;
coldline?: boolean;
dataLocations?: string[];
dra?: boolean;
enableObjectRetention?: boolean;
multiRegional?: boolean;
nearline?: boolean;
predefinedAcl?: 'authenticatedRead' | 'private' | 'projectPrivate' | 'publicRead' | 'publicReadWrite';
predefinedDefaultObjectAcl?: 'authenticatedRead' | 'bucketOwnerFullControl' | 'bucketOwnerRead' | 'private' | 'projectPrivate' | 'publicRead';
projection?: 'full' | 'noAcl';
regional?: boolean;
requesterPays?: boolean;
rpo?: string;
standard?: boolean;
storageClass?: string;
userProject?: string;
versioning?: Versioning;
}
export type CreateBucketResponse = [Bucket, unknown];
export interface BucketCallback {
(err: Error | null, bucket?: Bucket | null, apiResponse?: unknown): void;
}
export type GetBucketsResponse = [Bucket[], {}, unknown];
export interface GetBucketsCallback {
(err: Error | null, buckets: Bucket[], nextQuery?: {}, apiResponse?: unknown): void;
}
export interface GetBucketsRequest {
prefix?: string;
project?: string;
autoPaginate?: boolean;
maxApiCalls?: number;
maxResults?: number;
pageToken?: string;
userProject?: string;
softDeleted?: boolean;
generation?: number;
}
export interface HmacKeyResourceResponse {
metadata: HmacKeyMetadata;
secret: string;
}
export type CreateHmacKeyResponse = [HmacKey, string, HmacKeyResourceResponse];
export interface CreateHmacKeyOptions {
projectId?: string;
userProject?: string;
}
export interface CreateHmacKeyCallback {
(err: Error | null, hmacKey?: HmacKey | null, secret?: string | null, apiResponse?: HmacKeyResourceResponse): void;
}
export interface GetHmacKeysOptions {
projectId?: string;
serviceAccountEmail?: string;
showDeletedKeys?: boolean;
autoPaginate?: boolean;
maxApiCalls?: number;
maxResults?: number;
pageToken?: string;
userProject?: string;
}
export interface GetHmacKeysCallback {
(err: Error | null, hmacKeys: HmacKey[] | null, nextQuery?: {}, apiResponse?: unknown): void;
}
export declare enum ExceptionMessages {
EXPIRATION_DATE_INVALID = "The expiration date provided was invalid.",
EXPIRATION_DATE_PAST = "An expiration date cannot be in the past."
}
export declare enum StorageExceptionMessages {
BUCKET_NAME_REQUIRED = "A bucket name is needed to use Cloud Storage.",
BUCKET_NAME_REQUIRED_CREATE = "A name is required to create a bucket.",
HMAC_SERVICE_ACCOUNT = "The first argument must be a service account email to create an HMAC key.",
HMAC_ACCESS_ID = "An access ID is needed to create an HmacKey object."
}
export type GetHmacKeysResponse = [HmacKey[]];
export declare const PROTOCOL_REGEX: RegExp;
/**
* Default behavior: Automatically retry retriable server errors.
*
* @const {boolean}
*/
export declare const AUTO_RETRY_DEFAULT = true;
/**
* Default behavior: Only attempt to retry retriable errors 3 times.
*
* @const {number}
*/
export declare const MAX_RETRY_DEFAULT = 3;
/**
* Default behavior: Wait twice as long as previous retry before retrying.
*
* @const {number}
*/
export declare const RETRY_DELAY_MULTIPLIER_DEFAULT = 2;
/**
* Default behavior: If the operation doesn't succeed after 600 seconds,
* stop retrying.
*
* @const {number}
*/
export declare const TOTAL_TIMEOUT_DEFAULT = 600;
/**
* Default behavior: Wait no more than 64 seconds between retries.
*
* @const {number}
*/
export declare const MAX_RETRY_DELAY_DEFAULT = 64;
/**
* Returns true if the API request should be retried, given the error that was
* given the first time the request was attempted.
* @const
* @param {error} err - The API error to check if it is appropriate to retry.
* @return {boolean} True if the API request should be retried, false otherwise.
*/
export declare const RETRYABLE_ERR_FN_DEFAULT: (err?: ApiError) => boolean;
/*! Developer Documentation
*
* Invoke this method to create a new Storage object bound with pre-determined
* configuration options. For each object that can be created (e.g., a bucket),
* there is an equivalent static and instance method. While they are classes,
* they can be instantiated without use of the `new` keyword.
*/
/**
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share objects with other
* users and allow other users to access your buckets and objects.
*
* This object provides constants to refer to the three permission levels that
* can be granted to an entity:
*
* - `gcs.acl.OWNER_ROLE` - ("OWNER")
* - `gcs.acl.READER_ROLE` - ("READER")
* - `gcs.acl.WRITER_ROLE` - ("WRITER")
*
* See {@link https://cloud.google.com/storage/docs/access-control/lists| About Access Control Lists}
*
* @name Storage#acl
* @type {object}
* @property {string} OWNER_ROLE
* @property {string} READER_ROLE
* @property {string} WRITER_ROLE
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const albums = storage.bucket('albums');
*
* //-
* // Make all of the files currently in a bucket publicly readable.
* //-
* const options = {
* entity: 'allUsers',
* role: storage.acl.READER_ROLE
* };
*
* albums.acl.add(options, function(err, aclObject) {});
*
* //-
* // Make any new objects added to a bucket publicly readable.
* //-
* albums.acl.default.add(options, function(err, aclObject) {});
*
* //-
* // Grant a user ownership permissions to a bucket.
* //-
* albums.acl.add({
* entity: 'user-useremail@example.com',
* role: storage.acl.OWNER_ROLE
* }, function(err, aclObject) {});
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* albums.acl.add(options).then(function(data) {
* const aclObject = data[0];
* const apiResponse = data[1];
* });
* ```
*/
/**
* Get {@link Bucket} objects for all of the buckets in your project as
* a readable object stream.
*
* @method Storage#getBucketsStream
* @param {GetBucketsRequest} [query] Query object for listing buckets.
* @returns {ReadableStream} A readable stream that emits {@link Bucket}
* instances.
*
* @example
* ```
* storage.getBucketsStream()
* .on('error', console.error)
* .on('data', function(bucket) {
* // bucket is a Bucket object.
* })
* .on('end', function() {
* // All buckets retrieved.
* });
*
* //-
* // If you anticipate many results, you can end a stream early to prevent
* // unnecessary processing and API requests.
* //-
* storage.getBucketsStream()
* .on('data', function(bucket) {
* this.end();
* });
* ```
*/
/**
* Get {@link HmacKey} objects for all of the HMAC keys in the project in a
* readable object stream.
*
* @method Storage#getHmacKeysStream
* @param {GetHmacKeysOptions} [options] Configuration options.
* @returns {ReadableStream} A readable stream that emits {@link HmacKey}
* instances.
*
* @example
* ```
* storage.getHmacKeysStream()
* .on('error', console.error)
* .on('data', function(hmacKey) {
* // hmacKey is an HmacKey object.
* })
* .on('end', function() {
* // All HmacKey retrieved.
* });
*
* //-
* // If you anticipate many results, you can end a stream early to prevent
* // unnecessary processing and API requests.
* //-
* storage.getHmacKeysStream()
* .on('data', function(bucket) {
* this.end();
* });
* ```
*/
/**
* <h4>ACLs</h4>
* Cloud Storage uses access control lists (ACLs) to manage object and
* bucket access. ACLs are the mechanism you use to share files with other users
* and allow other users to access your buckets and files.
*
* To learn more about ACLs, read this overview on
* {@link https://cloud.google.com/storage/docs/access-control| Access Control}.
*
* See {@link https://cloud.google.com/storage/docs/overview| Cloud Storage overview}
* See {@link https://cloud.google.com/storage/docs/access-control| Access Control}
*
* @class
*/
export declare class Storage extends Service {
/**
* {@link Bucket} class.
*
* @name Storage.Bucket
* @see Bucket
* @type {Constructor}
*/
static Bucket: typeof Bucket;
/**
* {@link Channel} class.
*
* @name Storage.Channel
* @see Channel
* @type {Constructor}
*/
static Channel: typeof Channel;
/**
* {@link File} class.
*
* @name Storage.File
* @see File
* @type {Constructor}
*/
static File: typeof File;
/**
* {@link HmacKey} class.
*
* @name Storage.HmacKey
* @see HmacKey
* @type {Constructor}
*/
static HmacKey: typeof HmacKey;
static acl: {
OWNER_ROLE: string;
READER_ROLE: string;
WRITER_ROLE: string;
};
/**
* Reference to {@link Storage.acl}.
*
* @name Storage#acl
* @see Storage.acl
*/
acl: typeof Storage.acl;
crc32cGenerator: CRC32CValidatorGenerator;
getBucketsStream(): Readable;
getHmacKeysStream(): Readable;
retryOptions: RetryOptions;
/**
* @callback Crc32cGeneratorToStringCallback
* A method returning the CRC32C as a base64-encoded string.
*
* @returns {string}
*
* @example
* Hashing the string 'data' should return 'rth90Q=='
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.toString(); // 'rth90Q=='
* ```
**/
/**
* @callback Crc32cGeneratorValidateCallback
* A method validating a base64-encoded CRC32C string.
*
* @param {string} [value] base64-encoded CRC32C string to validate
* @returns {boolean}
*
* @example
* Should return `true` if the value matches, `false` otherwise
*
* ```js
* const buffer = Buffer.from('data');
* crc32c.update(buffer);
* crc32c.validate('DkjKuA=='); // false
* crc32c.validate('rth90Q=='); // true
* ```
**/
/**
* @callback Crc32cGeneratorUpdateCallback
* A method for passing `Buffer`s for CRC32C generation.
*
* @param {Buffer} [data] data to update CRC32C value with
* @returns {undefined}
*
* @example
* Hashing buffers from 'some ' and 'text\n'
*
* ```js
* const buffer1 = Buffer.from('some ');
* crc32c.update(buffer1);
*
* const buffer2 = Buffer.from('text\n');
* crc32c.update(buffer2);
*
* crc32c.toString(); // 'DkjKuA=='
* ```
**/
/**
* @typedef {object} CRC32CValidator
* @property {Crc32cGeneratorToStringCallback}
* @property {Crc32cGeneratorValidateCallback}
* @property {Crc32cGeneratorUpdateCallback}
*/
/**
* @callback Crc32cGeneratorCallback
* @returns {CRC32CValidator}
*/
/**
* @typedef {object} StorageOptions
* @property {string} [projectId] The project ID from the Google Developer's
* Console, e.g. 'grape-spaceship-123'. We will also check the environment
* variable `GCLOUD_PROJECT` for your project ID. If your app is running
* in an environment which supports {@link
* https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application
* Application Default Credentials}, your project ID will be detected
* automatically.
* @property {string} [keyFilename] Full path to the a .json, .pem, or .p12 key
* downloaded from the Google Developers Console. If you provide a path to
* a JSON file, the `projectId` option above is not necessary. NOTE: .pem and
* .p12 require you to specify the `email` option as well.
* @property {string} [email] Account email address. Required when using a .pem
* or .p12 keyFilename.
* @property {object} [credentials] Credentials object.
* @property {string} [credentials.client_email]
* @property {string} [credentials.private_key]
* @property {object} [retryOptions] Options for customizing retries. Retriable server errors
* will be retried with exponential delay between them dictated by the formula
* max(maxRetryDelay, retryDelayMultiplier*retryNumber) until maxRetries or totalTimeout
* has been reached. Retries will only happen if autoRetry is set to true.
* @property {boolean} [retryOptions.autoRetry=true] Automatically retry requests if the
* response is related to rate limits or certain intermittent server
* errors. We will exponentially backoff subsequent requests by default.
* @property {number} [retryOptions.retryDelayMultiplier = 2] the multiplier by which to
* increase the delay time between the completion of failed requests, and the
* initiation of the subsequent retrying request.
* @property {number} [retryOptions.totalTimeout = 600] The total time, starting from
* when the initial request is sent, after which an error will
* be returned, regardless of the retrying attempts made meanwhile.
* @property {number} [retryOptions.maxRetryDelay = 64] The maximum delay time between requests.
* When this value is reached, ``retryDelayMultiplier`` will no longer be used to
* increase delay time.
* @property {number} [retryOptions.maxRetries=3] Maximum number of automatic retries
* attempted before returning the error.
* @property {function} [retryOptions.retryableErrorFn] Function that returns true if a given
* error should be retried and false otherwise.
* @property {enum} [retryOptions.idempotencyStrategy=IdempotencyStrategy.RetryConditional] Enumeration
* controls how conditionally idempotent operations are retried. Possible values are: RetryAlways -
* will respect other retry settings and attempt to retry conditionally idempotent operations. RetryConditional -
* will retry conditionally idempotent operations if the correct preconditions are set. RetryNever - never
* retry a conditionally idempotent operation.
* @property {string} [userAgent] The value to be prepended to the User-Agent
* header in API requests.
* @property {object} [authClient] `AuthClient` or `GoogleAuth` client to reuse instead of creating a new one.
* @property {number} [timeout] The amount of time in milliseconds to wait per http request before timing out.
* @property {object[]} [interceptors_] Array of custom request interceptors to be returned in the order they were assigned.
* @property {string} [apiEndpoint = storage.google.com] The API endpoint of the service used to make requests.
* @property {boolean} [useAuthWithCustomEndpoint = false] Controls whether or not to use authentication when using a custom endpoint.
* @property {Crc32cGeneratorCallback} [callback] A function that generates a CRC32C Validator. Defaults to {@link CRC32C}
*/
/**
* Constructs the Storage client.
*
* @example
* Create a client that uses Application Default Credentials
* (ADC)
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* ```
*
* @example
* Create a client with explicit credentials
* ```
* const storage = new Storage({
* projectId: 'your-project-id',
* keyFilename: '/path/to/keyfile.json'
* });
* ```
*
* @example
* Create a client with credentials passed
* by value as a JavaScript object
* ```
* const storage = new Storage({
* projectId: 'your-project-id',
* credentials: {
* type: 'service_account',
* project_id: 'xxxxxxx',
* private_key_id: 'xxxx',
* private_key:'-----BEGIN PRIVATE KEY-----xxxxxxx\n-----END PRIVATE KEY-----\n',
* client_email: 'xxxx',
* client_id: 'xxx',
* auth_uri: 'https://accounts.google.com/o/oauth2/auth',
* token_uri: 'https://oauth2.googleapis.com/token',
* auth_provider_x509_cert_url: 'https://www.googleapis.com/oauth2/v1/certs',
* client_x509_cert_url: 'xxx',
* }
* });
* ```
*
* @example
* Create a client with credentials passed
* by loading a JSON file directly from disk
* ```
* const storage = new Storage({
* projectId: 'your-project-id',
* credentials: require('/path/to-keyfile.json')
* });
* ```
*
* @example
* Create a client with an `AuthClient` (e.g. `DownscopedClient`)
* ```
* const {DownscopedClient} = require('google-auth-library');
* const authClient = new DownscopedClient({...});
*
* const storage = new Storage({authClient});
* ```
*
* Additional samples:
* - https://github.com/googleapis/google-auth-library-nodejs#sample-usage-1
* - https://github.com/googleapis/google-auth-library-nodejs/blob/main/samples/downscopedclient.js
*
* @param {StorageOptions} [options] Configuration options.
*/
constructor(options?: StorageOptions);
private static sanitizeEndpoint;
/**
* Get a reference to a Cloud Storage bucket.
*
* @param {string} name Name of the bucket.
* @param {object} [options] Configuration object.
* @param {string} [options.kmsKeyName] A Cloud KMS key that will be used to
* encrypt objects inserted into this bucket, if no encryption method is
* specified.
* @param {string} [options.userProject] User project to be billed for all
* requests made from this Bucket object.
* @returns {Bucket}
* @see Bucket
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const albums = storage.bucket('albums');
* const photos = storage.bucket('photos');
* ```
*/
bucket(name: string, options?: BucketOptions): Bucket;
/**
* Reference a channel to receive notifications about changes to your bucket.
*
* @param {string} id The ID of the channel.
* @param {string} resourceId The resource ID of the channel.
* @returns {Channel}
* @see Channel
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const channel = storage.channel('id', 'resource-id');
* ```
*/
channel(id: string, resourceId: string): Channel;
createBucket(name: string, metadata?: CreateBucketRequest): Promise<CreateBucketResponse>;
createBucket(name: string, callback: BucketCallback): void;
createBucket(name: string, metadata: CreateBucketRequest, callback: BucketCallback): void;
createBucket(name: string, metadata: CreateBucketRequest, callback: BucketCallback): void;
createHmacKey(serviceAccountEmail: string, options?: CreateHmacKeyOptions): Promise<CreateHmacKeyResponse>;
createHmacKey(serviceAccountEmail: string, callback: CreateHmacKeyCallback): void;
createHmacKey(serviceAccountEmail: string, options: CreateHmacKeyOptions, callback: CreateHmacKeyCallback): void;
getBuckets(options?: GetBucketsRequest): Promise<GetBucketsResponse>;
getBuckets(options: GetBucketsRequest, callback: GetBucketsCallback): void;
getBuckets(callback: GetBucketsCallback): void;
/**
* Query object for listing HMAC keys.
*
* @typedef {object} GetHmacKeysOptions
* @property {string} [projectId] The project ID of the project that owns
* the service account of the requested HMAC key. If not provided,
* the project ID used to instantiate the Storage client will be used.
* @property {string} [serviceAccountEmail] If present, only HMAC keys for the
* given service account are returned.
* @property {boolean} [showDeletedKeys=false] If true, include keys in the DELETE
* state. Default is false.
* @property {boolean} [autoPaginate=true] Have pagination handled
* automatically.
* @property {number} [maxApiCalls] Maximum number of API calls to make.
* @property {number} [maxResults] Maximum number of items plus prefixes to
* return per call.
* Note: By default will handle pagination automatically
* if more than 1 page worth of results are requested per call.
* When `autoPaginate` is set to `false` the smaller of `maxResults`
* or 1 page of results will be returned per call.
* @property {string} [pageToken] A previously-returned page token
* representing part of the larger set of results to view.
* @property {string} [userProject] This parameter is currently ignored.
*/
/**
* @typedef {array} GetHmacKeysResponse
* @property {HmacKey[]} 0 Array of {@link HmacKey} instances.
* @param {object} nextQuery 1 A query object to receive more results.
* @param {object} apiResponse 2 The full API response.
*/
/**
* @callback GetHmacKeysCallback
* @param {?Error} err Request error, if any.
* @param {HmacKey[]} hmacKeys Array of {@link HmacKey} instances.
* @param {object} nextQuery A query object to receive more results.
* @param {object} apiResponse The full API response.
*/
/**
* Retrieves a list of HMAC keys matching the criteria.
*
* The authenticated user must have storage.hmacKeys.list permission for the project in which the key exists.
*
* @param {GetHmacKeysOption} options Configuration options.
* @param {GetHmacKeysCallback} callback Callback function.
* @return {Promise<GetHmacKeysResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* storage.getHmacKeys(function(err, hmacKeys) {
* if (!err) {
* // hmacKeys is an array of HmacKey objects.
* }
* });
*
* //-
* // To control how many API requests are made and page through the results
* // manually, set `autoPaginate` to `false`.
* //-
* const callback = function(err, hmacKeys, nextQuery, apiResponse) {
* if (nextQuery) {
* // More results exist.
* storage.getHmacKeys(nextQuery, callback);
* }
*
* // The `metadata` property is populated for you with the metadata at the
* // time of fetching.
* hmacKeys[0].metadata;
* };
*
* storage.getHmacKeys({
* autoPaginate: false
* }, callback);
*
* //-
* // If the callback is omitted, we'll return a Promise.
* //-
* storage.getHmacKeys().then(function(data) {
* const hmacKeys = data[0];
* });
* ```
*/
getHmacKeys(options?: GetHmacKeysOptions): Promise<GetHmacKeysResponse>;
getHmacKeys(callback: GetHmacKeysCallback): void;
getHmacKeys(options: GetHmacKeysOptions, callback: GetHmacKeysCallback): void;
getServiceAccount(options?: GetServiceAccountOptions): Promise<GetServiceAccountResponse>;
getServiceAccount(options?: GetServiceAccountOptions): Promise<GetServiceAccountResponse>;
getServiceAccount(options: GetServiceAccountOptions, callback: GetServiceAccountCallback): void;
getServiceAccount(callback: GetServiceAccountCallback): void;
/**
* Get a reference to an HmacKey object.
* Note: this does not fetch the HMAC key's metadata. Use HmacKey#get() to
* retrieve and populate the metadata.
*
* To get a reference to an HMAC key that's not created for a service
* account in the same project used to instantiate the Storage client,
* supply the project's ID as `projectId` in the `options` argument.
*
* @param {string} accessId The HMAC key's access ID.
* @param {HmacKeyOptions} options HmacKey constructor options.
* @returns {HmacKey}
* @see HmacKey
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const hmacKey = storage.hmacKey('ACCESS_ID');
* ```
*/
hmacKey(accessId: string, options?: HmacKeyOptions): HmacKey;
}
export {};

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,253 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { Bucket, UploadOptions, UploadResponse } from './bucket.js';
import { DownloadOptions, DownloadResponse, File } from './file.js';
import { GaxiosResponse } from 'gaxios';
export interface UploadManyFilesOptions {
concurrencyLimit?: number;
customDestinationBuilder?(path: string, options: UploadManyFilesOptions): string;
skipIfExists?: boolean;
prefix?: string;
passthroughOptions?: Omit<UploadOptions, 'destination'>;
}
export interface DownloadManyFilesOptions {
concurrencyLimit?: number;
prefix?: string;
stripPrefix?: string;
passthroughOptions?: DownloadOptions;
skipIfExists?: boolean;
}
export interface DownloadFileInChunksOptions {
concurrencyLimit?: number;
chunkSizeBytes?: number;
destination?: string;
validation?: 'crc32c' | false;
noReturnData?: boolean;
}
export interface UploadFileInChunksOptions {
concurrencyLimit?: number;
chunkSizeBytes?: number;
uploadName?: string;
maxQueueSize?: number;
uploadId?: string;
autoAbortFailure?: boolean;
partsMap?: Map<number, string>;
validation?: 'md5' | false;
headers?: {
[key: string]: string;
};
}
export interface MultiPartUploadHelper {
bucket: Bucket;
fileName: string;
uploadId?: string;
partsMap?: Map<number, string>;
initiateUpload(headers?: {
[key: string]: string;
}): Promise<void>;
uploadPart(partNumber: number, chunk: Buffer, validation?: 'md5' | false): Promise<void>;
completeUpload(): Promise<GaxiosResponse | undefined>;
abortUpload(): Promise<void>;
}
export type MultiPartHelperGenerator = (bucket: Bucket, fileName: string, uploadId?: string, partsMap?: Map<number, string>) => MultiPartUploadHelper;
export declare class MultiPartUploadError extends Error {
private uploadId;
private partsMap;
constructor(message: string, uploadId: string, partsMap: Map<number, string>);
}
/**
* Create a TransferManager object to perform parallel transfer operations on a Cloud Storage bucket.
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket A {@link Bucket} instance
*
*/
export declare class TransferManager {
bucket: Bucket;
constructor(bucket: Bucket);
/**
* @typedef {object} UploadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the files.
* @property {Function} [customDestinationBuilder] A function that will take the current path of a local file
* and return a string representing a custom path to be used to upload the file to GCS.
* @property {boolean} [skipIfExists] Do not upload the file if it already exists in
* the bucket. This will set the precondition ifGenerationMatch = 0.
* @property {string} [prefix] A prefix to append to all of the uploaded files.
* @property {object} [passthroughOptions] {@link UploadOptions} Options to be passed through
* to each individual upload operation.
*
*/
/**
* Upload multiple files in parallel to the bucket. This is a convenience method
* that utilizes {@link Bucket#upload} to perform the upload.
*
* @param {array | string} [filePathsOrDirectory] An array of fully qualified paths to the files or a directory name.
* If a directory name is provided, the directory will be recursively walked and all files will be added to the upload list.
* to be uploaded to the bucket
* @param {UploadManyFilesOptions} [options] Configuration options.
* @returns {Promise<UploadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload multiple files in parallel.
* //-
* const response = await transferManager.uploadManyFiles(['/local/path/file1.txt, 'local/path/file2.txt']);
* // Your bucket now contains:
* // - "local/path/file1.txt" (with the contents of '/local/path/file1.txt')
* // - "local/path/file2.txt" (with the contents of '/local/path/file2.txt')
* const response = await transferManager.uploadManyFiles('/local/directory');
* // Your bucket will now contain all files contained in '/local/directory' maintaining the subdirectory structure.
* ```
*
*/
uploadManyFiles(filePathsOrDirectory: string[] | string, options?: UploadManyFilesOptions): Promise<UploadResponse[]>;
/**
* @typedef {object} DownloadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the files.
* @property {string} [prefix] A prefix to append to all of the downloaded files.
* @property {string} [stripPrefix] A prefix to remove from all of the downloaded files.
* @property {object} [passthroughOptions] {@link DownloadOptions} Options to be passed through
* to each individual download operation.
* @property {boolean} [skipIfExists] Do not download the file if it already exists in
* the destination.
*
*/
/**
* Download multiple files in parallel to the local filesystem. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {array | string} [filesOrFolder] An array of file name strings or file objects to be downloaded. If
* a string is provided this will be treated as a GCS prefix and all files with that prefix will be downloaded.
* @param {DownloadManyFilesOptions} [options] Configuration options. Setting options.prefix or options.stripPrefix
* or options.passthroughOptions.destination will cause the downloaded files to be written to the file system
* instead of being returned as a buffer.
* @returns {Promise<DownloadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download multiple files in parallel.
* //-
* const response = await transferManager.downloadManyFiles(['file1.txt', 'file2.txt']);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles([bucket.File('file1.txt'), bucket.File('file2.txt')]);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles('test-folder');
* // All files with GCS prefix of 'test-folder' have been downloaded.
* ```
*
*/
downloadManyFiles(filesOrFolder: File[] | string[] | string, options?: DownloadManyFilesOptions): Promise<void | DownloadResponse[]>;
/**
* @typedef {object} DownloadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be downloaded.
* @property {string | boolean} [validation] Whether or not to perform a CRC32C validation check when download is complete.
* @property {boolean} [noReturnData] Whether or not to return the downloaded data. A `true` value here would be useful for files with a size that will not fit into memory.
*
*/
/**
* Download a large file in chunks utilizing parallel download operations. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {File | string} fileOrName {@link File} to download.
* @param {DownloadFileInChunksOptions} [options] Configuration options.
* @returns {Promise<void | DownloadResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.downloadFileInChunks(bucket.file('large-file.txt');
* // Your local directory now contains:
* // - "large-file.txt" (with the contents from my-bucket.large-file.txt)
* ```
*
*/
downloadFileInChunks(fileOrName: File | string, options?: DownloadFileInChunksOptions): Promise<void | DownloadResponse>;
/**
* @typedef {object} UploadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be uploaded.
* @property {string} [uploadName] Name of the file when saving to GCS. If omitted the name is taken from the file path.
* @property {number} [maxQueueSize] The number of chunks to be uploaded to hold in memory concurrently. If not specified
* defaults to the specified concurrency limit.
* @property {string} [uploadId] If specified attempts to resume a previous upload.
* @property {Map} [partsMap] If specified alongside uploadId, attempts to resume a previous upload from the last chunk
* specified in partsMap
* @property {object} [headers] headers to be sent when initiating the multipart upload.
* See {@link https://cloud.google.com/storage/docs/xml-api/post-object-multipart#request_headers| Request Headers: Initiate a Multipart Upload}
* @property {boolean} [autoAbortFailure] boolean to indicate if an in progress upload session will be automatically aborted upon failure. If not set,
* failures will be automatically aborted.
*
*/
/**
* Upload a large file in chunks utilizing parallel upload operations. If the upload fails, an uploadId and
* map containing all the successfully uploaded parts will be returned to the caller. These arguments can be used to
* resume the upload.
*
* @param {string} [filePath] The path of the file to be uploaded
* @param {UploadFileInChunksOptions} [options] Configuration options.
* @param {MultiPartHelperGenerator} [generator] A function that will return a type that implements the MPU interface. Most users will not need to use this.
* @returns {Promise<void>} If successful a promise resolving to void, otherwise a error containing the message, uploadId, and parts map.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.uploadFileInChunks('large-file.txt');
* // Your bucket now contains:
* // - "large-file.txt"
* ```
*
*
*/
uploadFileInChunks(filePath: string, options?: UploadFileInChunksOptions, generator?: MultiPartHelperGenerator): Promise<GaxiosResponse | undefined>;
private getPathsFromDirectory;
}

View File

@@ -0,0 +1,653 @@
/*!
* Copyright 2022 Google LLC. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _XMLMultiPartUploadHelper_instances, _XMLMultiPartUploadHelper_setGoogApiClientHeaders, _XMLMultiPartUploadHelper_handleErrorResponse;
import { FileExceptionMessages, RequestError, } from './file.js';
import pLimit from 'p-limit';
import * as path from 'path';
import { createReadStream, existsSync, promises as fsp } from 'fs';
import { CRC32C } from './crc32c.js';
import { GoogleAuth } from 'google-auth-library';
import { XMLParser, XMLBuilder } from 'fast-xml-parser';
import AsyncRetry from 'async-retry';
import { createHash } from 'crypto';
import { GCCL_GCS_CMD_KEY } from './nodejs-common/util.js';
import { getRuntimeTrackingString, getUserAgentString } from './util.js';
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
import { getPackageJSON } from './package-json-helper.cjs';
const packageJson = getPackageJSON();
/**
* Default number of concurrently executing promises to use when calling uploadManyFiles.
*
*/
const DEFAULT_PARALLEL_UPLOAD_LIMIT = 5;
/**
* Default number of concurrently executing promises to use when calling downloadManyFiles.
*
*/
const DEFAULT_PARALLEL_DOWNLOAD_LIMIT = 5;
/**
* Default number of concurrently executing promises to use when calling downloadFileInChunks.
*
*/
const DEFAULT_PARALLEL_CHUNKED_DOWNLOAD_LIMIT = 5;
/**
* The minimum size threshold in bytes at which to apply a chunked download strategy when calling downloadFileInChunks.
*
*/
const DOWNLOAD_IN_CHUNKS_FILE_SIZE_THRESHOLD = 32 * 1024 * 1024;
/**
* The chunk size in bytes to use when calling downloadFileInChunks.
*
*/
const DOWNLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE = 32 * 1024 * 1024;
/**
* The chunk size in bytes to use when calling uploadFileInChunks.
*
*/
const UPLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE = 32 * 1024 * 1024;
/**
* Default number of concurrently executing promises to use when calling uploadFileInChunks.
*
*/
const DEFAULT_PARALLEL_CHUNKED_UPLOAD_LIMIT = 5;
const EMPTY_REGEX = '(?:)';
/**
* The `gccl-gcs-cmd` value for the `X-Goog-API-Client` header.
* Example: `gccl-gcs-cmd/tm.upload_many`
*
* @see {@link GCCL_GCS_CMD}.
* @see {@link GCCL_GCS_CMD_KEY}.
*/
const GCCL_GCS_CMD_FEATURE = {
UPLOAD_MANY: 'tm.upload_many',
DOWNLOAD_MANY: 'tm.download_many',
UPLOAD_SHARDED: 'tm.upload_sharded',
DOWNLOAD_SHARDED: 'tm.download_sharded',
};
const defaultMultiPartGenerator = (bucket, fileName, uploadId, partsMap) => {
return new XMLMultiPartUploadHelper(bucket, fileName, uploadId, partsMap);
};
export class MultiPartUploadError extends Error {
constructor(message, uploadId, partsMap) {
super(message);
this.uploadId = uploadId;
this.partsMap = partsMap;
}
}
/**
* Class representing an implementation of MPU in the XML API. This class is not meant for public usage.
*
* @private
*
*/
class XMLMultiPartUploadHelper {
constructor(bucket, fileName, uploadId, partsMap) {
_XMLMultiPartUploadHelper_instances.add(this);
this.authClient = bucket.storage.authClient || new GoogleAuth();
this.uploadId = uploadId || '';
this.bucket = bucket;
this.fileName = fileName;
this.baseUrl = `https://${bucket.name}.${new URL(this.bucket.storage.apiEndpoint).hostname}/${fileName}`;
this.xmlBuilder = new XMLBuilder({ arrayNodeName: 'Part' });
this.xmlParser = new XMLParser();
this.partsMap = partsMap || new Map();
this.retryOptions = {
retries: this.bucket.storage.retryOptions.maxRetries,
factor: this.bucket.storage.retryOptions.retryDelayMultiplier,
maxTimeout: this.bucket.storage.retryOptions.maxRetryDelay * 1000,
maxRetryTime: this.bucket.storage.retryOptions.totalTimeout * 1000,
};
}
/**
* Initiates a multipart upload (MPU) to the XML API and stores the resultant upload id.
*
* @returns {Promise<void>}
*/
async initiateUpload(headers = {}) {
const url = `${this.baseUrl}?uploads`;
return AsyncRetry(async (bail) => {
try {
const res = await this.authClient.request({
headers: __classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_setGoogApiClientHeaders).call(this, headers),
method: 'POST',
url,
});
if (res.data && res.data.error) {
throw res.data.error;
}
const parsedXML = this.xmlParser.parse(res.data);
this.uploadId = parsedXML.InitiateMultipartUploadResult.UploadId;
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
}
}, this.retryOptions);
}
/**
* Uploads the provided chunk of data to the XML API using the previously created upload id.
*
* @param {number} partNumber the sequence number of this chunk.
* @param {Buffer} chunk the chunk of data to be uploaded.
* @param {string | false} validation whether or not to include the md5 hash in the headers to cause the server
* to validate the chunk was not corrupted.
* @returns {Promise<void>}
*/
async uploadPart(partNumber, chunk, validation) {
const url = `${this.baseUrl}?partNumber=${partNumber}&uploadId=${this.uploadId}`;
let headers = __classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_setGoogApiClientHeaders).call(this);
if (validation === 'md5') {
const hash = createHash('md5').update(chunk).digest('base64');
headers = {
'Content-MD5': hash,
};
}
return AsyncRetry(async (bail) => {
try {
const res = await this.authClient.request({
url,
method: 'PUT',
body: chunk,
headers,
});
if (res.data && res.data.error) {
throw res.data.error;
}
this.partsMap.set(partNumber, res.headers['etag']);
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
}
}, this.retryOptions);
}
/**
* Sends the final request of the MPU to tell GCS the upload is now complete.
*
* @returns {Promise<void>}
*/
async completeUpload() {
const url = `${this.baseUrl}?uploadId=${this.uploadId}`;
const sortedMap = new Map([...this.partsMap.entries()].sort((a, b) => a[0] - b[0]));
const parts = [];
for (const entry of sortedMap.entries()) {
parts.push({ PartNumber: entry[0], ETag: entry[1] });
}
const body = `<CompleteMultipartUpload>${this.xmlBuilder.build(parts)}</CompleteMultipartUpload>`;
return AsyncRetry(async (bail) => {
try {
const res = await this.authClient.request({
headers: __classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_setGoogApiClientHeaders).call(this),
url,
method: 'POST',
body,
});
if (res.data && res.data.error) {
throw res.data.error;
}
return res;
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
return;
}
}, this.retryOptions);
}
/**
* Aborts an multipart upload that is in progress. Once aborted, any parts in the process of being uploaded fail,
* and future requests using the upload ID fail.
*
* @returns {Promise<void>}
*/
async abortUpload() {
const url = `${this.baseUrl}?uploadId=${this.uploadId}`;
return AsyncRetry(async (bail) => {
try {
const res = await this.authClient.request({
url,
method: 'DELETE',
});
if (res.data && res.data.error) {
throw res.data.error;
}
}
catch (e) {
__classPrivateFieldGet(this, _XMLMultiPartUploadHelper_instances, "m", _XMLMultiPartUploadHelper_handleErrorResponse).call(this, e, bail);
return;
}
}, this.retryOptions);
}
}
_XMLMultiPartUploadHelper_instances = new WeakSet(), _XMLMultiPartUploadHelper_setGoogApiClientHeaders = function _XMLMultiPartUploadHelper_setGoogApiClientHeaders(headers = {}) {
let headerFound = false;
let userAgentFound = false;
for (const [key, value] of Object.entries(headers)) {
if (key.toLocaleLowerCase().trim() === 'x-goog-api-client') {
headerFound = true;
// Prepend command feature to value, if not already there
if (!value.includes(GCCL_GCS_CMD_FEATURE.UPLOAD_SHARDED)) {
headers[key] =
`${value} gccl-gcs-cmd/${GCCL_GCS_CMD_FEATURE.UPLOAD_SHARDED}`;
}
}
else if (key.toLocaleLowerCase().trim() === 'user-agent') {
userAgentFound = true;
}
}
// If the header isn't present, add it
if (!headerFound) {
headers['x-goog-api-client'] = `${getRuntimeTrackingString()} gccl/${packageJson.version} gccl-gcs-cmd/${GCCL_GCS_CMD_FEATURE.UPLOAD_SHARDED}`;
}
// If the User-Agent isn't present, add it
if (!userAgentFound) {
headers['User-Agent'] = getUserAgentString();
}
return headers;
}, _XMLMultiPartUploadHelper_handleErrorResponse = function _XMLMultiPartUploadHelper_handleErrorResponse(err, bail) {
if (this.bucket.storage.retryOptions.autoRetry &&
this.bucket.storage.retryOptions.retryableErrorFn(err)) {
throw err;
}
else {
bail(err);
}
};
/**
* Create a TransferManager object to perform parallel transfer operations on a Cloud Storage bucket.
*
* @class
* @hideconstructor
*
* @param {Bucket} bucket A {@link Bucket} instance
*
*/
export class TransferManager {
constructor(bucket) {
this.bucket = bucket;
}
/**
* @typedef {object} UploadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the files.
* @property {Function} [customDestinationBuilder] A function that will take the current path of a local file
* and return a string representing a custom path to be used to upload the file to GCS.
* @property {boolean} [skipIfExists] Do not upload the file if it already exists in
* the bucket. This will set the precondition ifGenerationMatch = 0.
* @property {string} [prefix] A prefix to append to all of the uploaded files.
* @property {object} [passthroughOptions] {@link UploadOptions} Options to be passed through
* to each individual upload operation.
*
*/
/**
* Upload multiple files in parallel to the bucket. This is a convenience method
* that utilizes {@link Bucket#upload} to perform the upload.
*
* @param {array | string} [filePathsOrDirectory] An array of fully qualified paths to the files or a directory name.
* If a directory name is provided, the directory will be recursively walked and all files will be added to the upload list.
* to be uploaded to the bucket
* @param {UploadManyFilesOptions} [options] Configuration options.
* @returns {Promise<UploadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload multiple files in parallel.
* //-
* const response = await transferManager.uploadManyFiles(['/local/path/file1.txt, 'local/path/file2.txt']);
* // Your bucket now contains:
* // - "local/path/file1.txt" (with the contents of '/local/path/file1.txt')
* // - "local/path/file2.txt" (with the contents of '/local/path/file2.txt')
* const response = await transferManager.uploadManyFiles('/local/directory');
* // Your bucket will now contain all files contained in '/local/directory' maintaining the subdirectory structure.
* ```
*
*/
async uploadManyFiles(filePathsOrDirectory, options = {}) {
var _a;
if (options.skipIfExists && ((_a = options.passthroughOptions) === null || _a === void 0 ? void 0 : _a.preconditionOpts)) {
options.passthroughOptions.preconditionOpts.ifGenerationMatch = 0;
}
else if (options.skipIfExists &&
options.passthroughOptions === undefined) {
options.passthroughOptions = {
preconditionOpts: {
ifGenerationMatch: 0,
},
};
}
const limit = pLimit(options.concurrencyLimit || DEFAULT_PARALLEL_UPLOAD_LIMIT);
const promises = [];
let allPaths = [];
if (!Array.isArray(filePathsOrDirectory)) {
for await (const curPath of this.getPathsFromDirectory(filePathsOrDirectory)) {
allPaths.push(curPath);
}
}
else {
allPaths = filePathsOrDirectory;
}
for (const filePath of allPaths) {
const stat = await fsp.lstat(filePath);
if (stat.isDirectory()) {
continue;
}
const passThroughOptionsCopy = {
...options.passthroughOptions,
[GCCL_GCS_CMD_KEY]: GCCL_GCS_CMD_FEATURE.UPLOAD_MANY,
};
passThroughOptionsCopy.destination = options.customDestinationBuilder
? options.customDestinationBuilder(filePath, options)
: filePath.split(path.sep).join(path.posix.sep);
if (options.prefix) {
passThroughOptionsCopy.destination = path.posix.join(...options.prefix.split(path.sep), passThroughOptionsCopy.destination);
}
promises.push(limit(() => this.bucket.upload(filePath, passThroughOptionsCopy)));
}
return Promise.all(promises);
}
/**
* @typedef {object} DownloadManyFilesOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the files.
* @property {string} [prefix] A prefix to append to all of the downloaded files.
* @property {string} [stripPrefix] A prefix to remove from all of the downloaded files.
* @property {object} [passthroughOptions] {@link DownloadOptions} Options to be passed through
* to each individual download operation.
* @property {boolean} [skipIfExists] Do not download the file if it already exists in
* the destination.
*
*/
/**
* Download multiple files in parallel to the local filesystem. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {array | string} [filesOrFolder] An array of file name strings or file objects to be downloaded. If
* a string is provided this will be treated as a GCS prefix and all files with that prefix will be downloaded.
* @param {DownloadManyFilesOptions} [options] Configuration options. Setting options.prefix or options.stripPrefix
* or options.passthroughOptions.destination will cause the downloaded files to be written to the file system
* instead of being returned as a buffer.
* @returns {Promise<DownloadResponse[]>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download multiple files in parallel.
* //-
* const response = await transferManager.downloadManyFiles(['file1.txt', 'file2.txt']);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles([bucket.File('file1.txt'), bucket.File('file2.txt')]);
* // The following files have been downloaded:
* // - "file1.txt" (with the contents from my-bucket.file1.txt)
* // - "file2.txt" (with the contents from my-bucket.file2.txt)
* const response = await transferManager.downloadManyFiles('test-folder');
* // All files with GCS prefix of 'test-folder' have been downloaded.
* ```
*
*/
async downloadManyFiles(filesOrFolder, options = {}) {
const limit = pLimit(options.concurrencyLimit || DEFAULT_PARALLEL_DOWNLOAD_LIMIT);
const promises = [];
let files = [];
if (!Array.isArray(filesOrFolder)) {
const directoryFiles = await this.bucket.getFiles({
prefix: filesOrFolder,
});
files = directoryFiles[0];
}
else {
files = filesOrFolder.map(curFile => {
if (typeof curFile === 'string') {
return this.bucket.file(curFile);
}
return curFile;
});
}
const stripRegexString = options.stripPrefix
? `^${options.stripPrefix}`
: EMPTY_REGEX;
const regex = new RegExp(stripRegexString, 'g');
for (const file of files) {
const passThroughOptionsCopy = {
...options.passthroughOptions,
[GCCL_GCS_CMD_KEY]: GCCL_GCS_CMD_FEATURE.DOWNLOAD_MANY,
};
if (options.prefix || passThroughOptionsCopy.destination) {
passThroughOptionsCopy.destination = path.join(options.prefix || '', passThroughOptionsCopy.destination || '', file.name);
}
if (options.stripPrefix) {
passThroughOptionsCopy.destination = file.name.replace(regex, '');
}
if (options.skipIfExists &&
existsSync(passThroughOptionsCopy.destination || '')) {
continue;
}
promises.push(limit(async () => {
const destination = passThroughOptionsCopy.destination;
if (destination && destination.endsWith(path.sep)) {
await fsp.mkdir(destination, { recursive: true });
return Promise.resolve([
Buffer.alloc(0),
]);
}
return file.download(passThroughOptionsCopy);
}));
}
return Promise.all(promises);
}
/**
* @typedef {object} DownloadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when downloading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be downloaded.
* @property {string | boolean} [validation] Whether or not to perform a CRC32C validation check when download is complete.
* @property {boolean} [noReturnData] Whether or not to return the downloaded data. A `true` value here would be useful for files with a size that will not fit into memory.
*
*/
/**
* Download a large file in chunks utilizing parallel download operations. This is a convenience method
* that utilizes {@link File#download} to perform the download.
*
* @param {File | string} fileOrName {@link File} to download.
* @param {DownloadFileInChunksOptions} [options] Configuration options.
* @returns {Promise<void | DownloadResponse>}
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Download a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.downloadFileInChunks(bucket.file('large-file.txt');
* // Your local directory now contains:
* // - "large-file.txt" (with the contents from my-bucket.large-file.txt)
* ```
*
*/
async downloadFileInChunks(fileOrName, options = {}) {
let chunkSize = options.chunkSizeBytes || DOWNLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE;
let limit = pLimit(options.concurrencyLimit || DEFAULT_PARALLEL_CHUNKED_DOWNLOAD_LIMIT);
const noReturnData = Boolean(options.noReturnData);
const promises = [];
const file = typeof fileOrName === 'string'
? this.bucket.file(fileOrName)
: fileOrName;
const fileInfo = await file.get();
const size = parseInt(fileInfo[0].metadata.size.toString());
// If the file size does not meet the threshold download it as a single chunk.
if (size < DOWNLOAD_IN_CHUNKS_FILE_SIZE_THRESHOLD) {
limit = pLimit(1);
chunkSize = size;
}
let start = 0;
const filePath = options.destination || path.basename(file.name);
const fileToWrite = await fsp.open(filePath, 'w');
while (start < size) {
const chunkStart = start;
let chunkEnd = start + chunkSize - 1;
chunkEnd = chunkEnd > size ? size : chunkEnd;
promises.push(limit(async () => {
const resp = await file.download({
start: chunkStart,
end: chunkEnd,
[GCCL_GCS_CMD_KEY]: GCCL_GCS_CMD_FEATURE.DOWNLOAD_SHARDED,
});
const result = await fileToWrite.write(resp[0], 0, resp[0].length, chunkStart);
if (noReturnData)
return;
return result.buffer;
}));
start += chunkSize;
}
let chunks;
try {
chunks = await Promise.all(promises);
}
finally {
await fileToWrite.close();
}
if (options.validation === 'crc32c' && fileInfo[0].metadata.crc32c) {
const downloadedCrc32C = await CRC32C.fromFile(filePath);
if (!downloadedCrc32C.validate(fileInfo[0].metadata.crc32c)) {
const mismatchError = new RequestError(FileExceptionMessages.DOWNLOAD_MISMATCH);
mismatchError.code = 'CONTENT_DOWNLOAD_MISMATCH';
throw mismatchError;
}
}
if (noReturnData)
return;
return [Buffer.concat(chunks, size)];
}
/**
* @typedef {object} UploadFileInChunksOptions
* @property {number} [concurrencyLimit] The number of concurrently executing promises
* to use when uploading the file.
* @property {number} [chunkSizeBytes] The size in bytes of each chunk to be uploaded.
* @property {string} [uploadName] Name of the file when saving to GCS. If omitted the name is taken from the file path.
* @property {number} [maxQueueSize] The number of chunks to be uploaded to hold in memory concurrently. If not specified
* defaults to the specified concurrency limit.
* @property {string} [uploadId] If specified attempts to resume a previous upload.
* @property {Map} [partsMap] If specified alongside uploadId, attempts to resume a previous upload from the last chunk
* specified in partsMap
* @property {object} [headers] headers to be sent when initiating the multipart upload.
* See {@link https://cloud.google.com/storage/docs/xml-api/post-object-multipart#request_headers| Request Headers: Initiate a Multipart Upload}
* @property {boolean} [autoAbortFailure] boolean to indicate if an in progress upload session will be automatically aborted upon failure. If not set,
* failures will be automatically aborted.
*
*/
/**
* Upload a large file in chunks utilizing parallel upload operations. If the upload fails, an uploadId and
* map containing all the successfully uploaded parts will be returned to the caller. These arguments can be used to
* resume the upload.
*
* @param {string} [filePath] The path of the file to be uploaded
* @param {UploadFileInChunksOptions} [options] Configuration options.
* @param {MultiPartHelperGenerator} [generator] A function that will return a type that implements the MPU interface. Most users will not need to use this.
* @returns {Promise<void>} If successful a promise resolving to void, otherwise a error containing the message, uploadId, and parts map.
*
* @example
* ```
* const {Storage} = require('@google-cloud/storage');
* const storage = new Storage();
* const bucket = storage.bucket('my-bucket');
* const transferManager = new TransferManager(bucket);
*
* //-
* // Upload a large file in chunks utilizing parallel operations.
* //-
* const response = await transferManager.uploadFileInChunks('large-file.txt');
* // Your bucket now contains:
* // - "large-file.txt"
* ```
*
*
*/
async uploadFileInChunks(filePath, options = {}, generator = defaultMultiPartGenerator) {
const chunkSize = options.chunkSizeBytes || UPLOAD_IN_CHUNKS_DEFAULT_CHUNK_SIZE;
const limit = pLimit(options.concurrencyLimit || DEFAULT_PARALLEL_CHUNKED_UPLOAD_LIMIT);
const maxQueueSize = options.maxQueueSize ||
options.concurrencyLimit ||
DEFAULT_PARALLEL_CHUNKED_UPLOAD_LIMIT;
const fileName = options.uploadName || path.basename(filePath);
const mpuHelper = generator(this.bucket, fileName, options.uploadId, options.partsMap);
let partNumber = 1;
let promises = [];
try {
if (options.uploadId === undefined) {
await mpuHelper.initiateUpload(options.headers);
}
const startOrResumptionByte = mpuHelper.partsMap.size * chunkSize;
const readStream = createReadStream(filePath, {
highWaterMark: chunkSize,
start: startOrResumptionByte,
});
// p-limit only limits the number of running promises. We do not want to hold an entire
// large file in memory at once so promises acts a queue that will hold only maxQueueSize in memory.
for await (const curChunk of readStream) {
if (promises.length >= maxQueueSize) {
await Promise.all(promises);
promises = [];
}
promises.push(limit(() => mpuHelper.uploadPart(partNumber++, curChunk, options.validation)));
}
await Promise.all(promises);
return await mpuHelper.completeUpload();
}
catch (e) {
if ((options.autoAbortFailure === undefined || options.autoAbortFailure) &&
mpuHelper.uploadId) {
try {
await mpuHelper.abortUpload();
return;
}
catch (e) {
throw new MultiPartUploadError(e.message, mpuHelper.uploadId, mpuHelper.partsMap);
}
}
throw new MultiPartUploadError(e.message, mpuHelper.uploadId, mpuHelper.partsMap);
}
}
async *getPathsFromDirectory(directory) {
const filesAndSubdirectories = await fsp.readdir(directory, {
withFileTypes: true,
});
for (const curFileOrDirectory of filesAndSubdirectories) {
const fullPath = path.join(directory, curFileOrDirectory.name);
curFileOrDirectory.isDirectory()
? yield* this.getPathsFromDirectory(fullPath)
: yield fullPath;
}
}
}

View File

@@ -0,0 +1,85 @@
import * as querystring from 'querystring';
import { PassThrough } from 'stream';
export declare function normalize<T = {}, U = Function>(optionsOrCallback?: T | U, cb?: U): {
options: T;
callback: U;
};
/**
* Flatten an object into an Array of arrays, [[key, value], ..].
* Implements Object.entries() for Node.js <8
* @internal
*/
export declare function objectEntries<T>(obj: {
[key: string]: T;
}): Array<[string, T]>;
/**
* Encode `str` with encodeURIComponent, plus these
* reserved characters: `! * ' ( )`.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent| MDN: fixedEncodeURIComponent}
*
* @param {string} str The URI component to encode.
* @return {string} The encoded string.
*/
export declare function fixedEncodeURIComponent(str: string): string;
/**
* URI encode `uri` for generating signed URLs, using fixedEncodeURIComponent.
*
* Encode every byte except `A-Z a-Z 0-9 ~ - . _`.
*
* @param {string} uri The URI to encode.
* @param [boolean=false] encodeSlash If `true`, the "/" character is not encoded.
* @return {string} The encoded string.
*/
export declare function encodeURI(uri: string, encodeSlash: boolean): string;
/**
* Serialize an object to a URL query string using util.encodeURI(uri, true).
* @param {string} url The object to serialize.
* @return {string} Serialized string.
*/
export declare function qsStringify(qs: querystring.ParsedUrlQueryInput): string;
export declare function objectKeyToLowercase<T>(object: {
[key: string]: T;
}): {
[key: string]: T;
};
/**
* JSON encode str, with unicode \u+ representation.
* @param {object} obj The object to encode.
* @return {string} Serialized string.
*/
export declare function unicodeJSONStringify(obj: object): string;
/**
* Converts the given objects keys to snake_case
* @param {object} obj object to convert keys to snake case.
* @returns {object} object with keys converted to snake case.
*/
export declare function convertObjKeysToSnakeCase(obj: object): object;
/**
* Formats the provided date object as a UTC ISO string.
* @param {Date} dateTimeToFormat date object to be formatted.
* @param {boolean} includeTime flag to include hours, minutes, seconds in output.
* @param {string} dateDelimiter delimiter between date components.
* @param {string} timeDelimiter delimiter between time components.
* @returns {string} UTC ISO format of provided date object.
*/
export declare function formatAsUTCISO(dateTimeToFormat: Date, includeTime?: boolean, dateDelimiter?: string, timeDelimiter?: string): string;
/**
* Examines the runtime environment and returns the appropriate tracking string.
* @returns {string} metrics tracking string based on the current runtime environment.
*/
export declare function getRuntimeTrackingString(): string;
/**
* Looks at package.json and creates the user-agent string to be applied to request headers.
* @returns {string} user agent string.
*/
export declare function getUserAgentString(): string;
export declare function getDirName(): string;
export declare function getModuleFormat(): "ESM" | "CJS";
export declare class PassThroughShim extends PassThrough {
private shouldEmitReading;
private shouldEmitWriting;
_read(size: number): void;
_write(chunk: never, encoding: BufferEncoding, callback: (error?: Error | null | undefined) => void): void;
_final(callback: (error?: Error | null | undefined) => void): void;
}

View File

@@ -0,0 +1,229 @@
// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import * as path from 'path';
import * as querystring from 'querystring';
import { PassThrough } from 'stream';
import * as url from 'url';
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
import { getPackageJSON } from './package-json-helper.cjs';
// Done to avoid a problem with mangling of identifiers when using esModuleInterop
const fileURLToPath = url.fileURLToPath;
const isEsm = true;
export function normalize(optionsOrCallback, cb) {
const options = (typeof optionsOrCallback === 'object' ? optionsOrCallback : {});
const callback = (typeof optionsOrCallback === 'function' ? optionsOrCallback : cb);
return { options, callback };
}
/**
* Flatten an object into an Array of arrays, [[key, value], ..].
* Implements Object.entries() for Node.js <8
* @internal
*/
export function objectEntries(obj) {
return Object.keys(obj).map(key => [key, obj[key]]);
}
/**
* Encode `str` with encodeURIComponent, plus these
* reserved characters: `! * ' ( )`.
*
* See {@link https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent| MDN: fixedEncodeURIComponent}
*
* @param {string} str The URI component to encode.
* @return {string} The encoded string.
*/
export function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, c => '%' + c.charCodeAt(0).toString(16).toUpperCase());
}
/**
* URI encode `uri` for generating signed URLs, using fixedEncodeURIComponent.
*
* Encode every byte except `A-Z a-Z 0-9 ~ - . _`.
*
* @param {string} uri The URI to encode.
* @param [boolean=false] encodeSlash If `true`, the "/" character is not encoded.
* @return {string} The encoded string.
*/
export function encodeURI(uri, encodeSlash) {
// Split the string by `/`, and conditionally rejoin them with either
// %2F if encodeSlash is `true`, or '/' if `false`.
return uri
.split('/')
.map(fixedEncodeURIComponent)
.join(encodeSlash ? '%2F' : '/');
}
/**
* Serialize an object to a URL query string using util.encodeURI(uri, true).
* @param {string} url The object to serialize.
* @return {string} Serialized string.
*/
export function qsStringify(qs) {
return querystring.stringify(qs, '&', '=', {
encodeURIComponent: (component) => encodeURI(component, true),
});
}
export function objectKeyToLowercase(object) {
const newObj = {};
for (let key of Object.keys(object)) {
const value = object[key];
key = key.toLowerCase();
newObj[key] = value;
}
return newObj;
}
/**
* JSON encode str, with unicode \u+ representation.
* @param {object} obj The object to encode.
* @return {string} Serialized string.
*/
export function unicodeJSONStringify(obj) {
return JSON.stringify(obj).replace(/[\u0080-\uFFFF]/g, (char) => '\\u' + ('0000' + char.charCodeAt(0).toString(16)).slice(-4));
}
/**
* Converts the given objects keys to snake_case
* @param {object} obj object to convert keys to snake case.
* @returns {object} object with keys converted to snake case.
*/
export function convertObjKeysToSnakeCase(obj) {
if (obj instanceof Date || obj instanceof RegExp) {
return obj;
}
if (Array.isArray(obj)) {
return obj.map(convertObjKeysToSnakeCase);
}
if (obj instanceof Object) {
return Object.keys(obj).reduce((acc, cur) => {
const s = cur[0].toLocaleLowerCase() +
cur.slice(1).replace(/([A-Z]+)/g, (match, p1) => {
return `_${p1.toLowerCase()}`;
});
acc[s] = convertObjKeysToSnakeCase(obj[cur]);
return acc;
}, Object());
}
return obj;
}
/**
* Formats the provided date object as a UTC ISO string.
* @param {Date} dateTimeToFormat date object to be formatted.
* @param {boolean} includeTime flag to include hours, minutes, seconds in output.
* @param {string} dateDelimiter delimiter between date components.
* @param {string} timeDelimiter delimiter between time components.
* @returns {string} UTC ISO format of provided date object.
*/
export function formatAsUTCISO(dateTimeToFormat, includeTime = false, dateDelimiter = '', timeDelimiter = '') {
const year = dateTimeToFormat.getUTCFullYear();
const month = dateTimeToFormat.getUTCMonth() + 1;
const day = dateTimeToFormat.getUTCDate();
const hour = dateTimeToFormat.getUTCHours();
const minute = dateTimeToFormat.getUTCMinutes();
const second = dateTimeToFormat.getUTCSeconds();
let resultString = `${year.toString().padStart(4, '0')}${dateDelimiter}${month
.toString()
.padStart(2, '0')}${dateDelimiter}${day.toString().padStart(2, '0')}`;
if (includeTime) {
resultString = `${resultString}T${hour
.toString()
.padStart(2, '0')}${timeDelimiter}${minute
.toString()
.padStart(2, '0')}${timeDelimiter}${second.toString().padStart(2, '0')}Z`;
}
return resultString;
}
/**
* Examines the runtime environment and returns the appropriate tracking string.
* @returns {string} metrics tracking string based on the current runtime environment.
*/
export function getRuntimeTrackingString() {
if (
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
globalThis.Deno &&
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
globalThis.Deno.version &&
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
globalThis.Deno.version.deno) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
return `gl-deno/${globalThis.Deno.version.deno}`;
}
else {
return `gl-node/${process.versions.node}`;
}
}
/**
* Looks at package.json and creates the user-agent string to be applied to request headers.
* @returns {string} user agent string.
*/
export function getUserAgentString() {
const pkg = getPackageJSON();
const hyphenatedPackageName = pkg.name
.replace('@google-cloud', 'gcloud-node') // For legacy purposes.
.replace('/', '-'); // For UA spec-compliance purposes.
return hyphenatedPackageName + '/' + pkg.version;
}
export function getDirName() {
let dirToUse = '';
try {
dirToUse = __dirname;
}
catch (e) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
dirToUse = path.dirname(fileURLToPath(import.meta.url));
}
return dirToUse;
}
export function getModuleFormat() {
return isEsm ? 'ESM' : 'CJS';
}
export class PassThroughShim extends PassThrough {
constructor() {
super(...arguments);
this.shouldEmitReading = true;
this.shouldEmitWriting = true;
}
_read(size) {
if (this.shouldEmitReading) {
this.emit('reading');
this.shouldEmitReading = false;
}
super._read(size);
}
_write(chunk, encoding, callback) {
if (this.shouldEmitWriting) {
this.emit('writing');
this.shouldEmitWriting = false;
}
// Per the nodejs documentation, callback must be invoked on the next tick
process.nextTick(() => {
super._write(chunk, encoding, callback);
});
}
_final(callback) {
// If the stream is empty (i.e. empty file) final will be invoked before _read / _write
// and we should still emit the proper events.
if (this.shouldEmitReading) {
this.emit('reading');
this.shouldEmitReading = false;
}
if (this.shouldEmitWriting) {
this.emit('writing');
this.shouldEmitWriting = false;
}
callback(null);
}
}

View File

@@ -0,0 +1 @@
../uuid/dist/bin/uuid

View File

@@ -0,0 +1,229 @@
# Changelog
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [8.3.2](https://github.com/uuidjs/uuid/compare/v8.3.1...v8.3.2) (2020-12-08)
### Bug Fixes
- lazy load getRandomValues ([#537](https://github.com/uuidjs/uuid/issues/537)) ([16c8f6d](https://github.com/uuidjs/uuid/commit/16c8f6df2f6b09b4d6235602d6a591188320a82e)), closes [#536](https://github.com/uuidjs/uuid/issues/536)
### [8.3.1](https://github.com/uuidjs/uuid/compare/v8.3.0...v8.3.1) (2020-10-04)
### Bug Fixes
- support expo>=39.0.0 ([#515](https://github.com/uuidjs/uuid/issues/515)) ([c65a0f3](https://github.com/uuidjs/uuid/commit/c65a0f3fa73b901959d638d1e3591dfacdbed867)), closes [#375](https://github.com/uuidjs/uuid/issues/375)
## [8.3.0](https://github.com/uuidjs/uuid/compare/v8.2.0...v8.3.0) (2020-07-27)
### Features
- add parse/stringify/validate/version/NIL APIs ([#479](https://github.com/uuidjs/uuid/issues/479)) ([0e6c10b](https://github.com/uuidjs/uuid/commit/0e6c10ba1bf9517796ff23c052fc0468eedfd5f4)), closes [#475](https://github.com/uuidjs/uuid/issues/475) [#478](https://github.com/uuidjs/uuid/issues/478) [#480](https://github.com/uuidjs/uuid/issues/480) [#481](https://github.com/uuidjs/uuid/issues/481) [#180](https://github.com/uuidjs/uuid/issues/180)
## [8.2.0](https://github.com/uuidjs/uuid/compare/v8.1.0...v8.2.0) (2020-06-23)
### Features
- improve performance of v1 string representation ([#453](https://github.com/uuidjs/uuid/issues/453)) ([0ee0b67](https://github.com/uuidjs/uuid/commit/0ee0b67c37846529c66089880414d29f3ae132d5))
- remove deprecated v4 string parameter ([#454](https://github.com/uuidjs/uuid/issues/454)) ([88ce3ca](https://github.com/uuidjs/uuid/commit/88ce3ca0ba046f60856de62c7ce03f7ba98ba46c)), closes [#437](https://github.com/uuidjs/uuid/issues/437)
- support jspm ([#473](https://github.com/uuidjs/uuid/issues/473)) ([e9f2587](https://github.com/uuidjs/uuid/commit/e9f2587a92575cac31bc1d4ae944e17c09756659))
### Bug Fixes
- prepare package exports for webpack 5 ([#468](https://github.com/uuidjs/uuid/issues/468)) ([8d6e6a5](https://github.com/uuidjs/uuid/commit/8d6e6a5f8965ca9575eb4d92e99a43435f4a58a8))
## [8.1.0](https://github.com/uuidjs/uuid/compare/v8.0.0...v8.1.0) (2020-05-20)
### Features
- improve v4 performance by reusing random number array ([#435](https://github.com/uuidjs/uuid/issues/435)) ([bf4af0d](https://github.com/uuidjs/uuid/commit/bf4af0d711b4d2ed03d1f74fd12ad0baa87dc79d))
- optimize V8 performance of bytesToUuid ([#434](https://github.com/uuidjs/uuid/issues/434)) ([e156415](https://github.com/uuidjs/uuid/commit/e156415448ec1af2351fa0b6660cfb22581971f2))
### Bug Fixes
- export package.json required by react-native and bundlers ([#449](https://github.com/uuidjs/uuid/issues/449)) ([be1c8fe](https://github.com/uuidjs/uuid/commit/be1c8fe9a3206c358e0059b52fafd7213aa48a52)), closes [ai/nanoevents#44](https://github.com/ai/nanoevents/issues/44#issuecomment-602010343) [#444](https://github.com/uuidjs/uuid/issues/444)
## [8.0.0](https://github.com/uuidjs/uuid/compare/v7.0.3...v8.0.0) (2020-04-29)
### ⚠ BREAKING CHANGES
- For native ECMAScript Module (ESM) usage in Node.js only named exports are exposed, there is no more default export.
```diff
-import uuid from 'uuid';
-console.log(uuid.v4()); // -> 'cd6c3b08-0adc-4f4b-a6ef-36087a1c9869'
+import { v4 as uuidv4 } from 'uuid';
+uuidv4(); // ⇨ '9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d'
```
- Deep requiring specific algorithms of this library like `require('uuid/v4')`, which has been deprecated in `uuid@7`, is no longer supported.
Instead use the named exports that this module exports.
For ECMAScript Modules (ESM):
```diff
-import uuidv4 from 'uuid/v4';
+import { v4 as uuidv4 } from 'uuid';
uuidv4();
```
For CommonJS:
```diff
-const uuidv4 = require('uuid/v4');
+const { v4: uuidv4 } = require('uuid');
uuidv4();
```
### Features
- native Node.js ES Modules (wrapper approach) ([#423](https://github.com/uuidjs/uuid/issues/423)) ([2d9f590](https://github.com/uuidjs/uuid/commit/2d9f590ad9701d692625c07ed62f0a0f91227991)), closes [#245](https://github.com/uuidjs/uuid/issues/245) [#419](https://github.com/uuidjs/uuid/issues/419) [#342](https://github.com/uuidjs/uuid/issues/342)
- remove deep requires ([#426](https://github.com/uuidjs/uuid/issues/426)) ([daf72b8](https://github.com/uuidjs/uuid/commit/daf72b84ceb20272a81bb5fbddb05dd95922cbba))
### Bug Fixes
- add CommonJS syntax example to README quickstart section ([#417](https://github.com/uuidjs/uuid/issues/417)) ([e0ec840](https://github.com/uuidjs/uuid/commit/e0ec8402c7ad44b7ef0453036c612f5db513fda0))
### [7.0.3](https://github.com/uuidjs/uuid/compare/v7.0.2...v7.0.3) (2020-03-31)
### Bug Fixes
- make deep require deprecation warning work in browsers ([#409](https://github.com/uuidjs/uuid/issues/409)) ([4b71107](https://github.com/uuidjs/uuid/commit/4b71107d8c0d2ef56861ede6403fc9dc35a1e6bf)), closes [#408](https://github.com/uuidjs/uuid/issues/408)
### [7.0.2](https://github.com/uuidjs/uuid/compare/v7.0.1...v7.0.2) (2020-03-04)
### Bug Fixes
- make access to msCrypto consistent ([#393](https://github.com/uuidjs/uuid/issues/393)) ([8bf2a20](https://github.com/uuidjs/uuid/commit/8bf2a20f3565df743da7215eebdbada9d2df118c))
- simplify link in deprecation warning ([#391](https://github.com/uuidjs/uuid/issues/391)) ([bb2c8e4](https://github.com/uuidjs/uuid/commit/bb2c8e4e9f4c5f9c1eaaf3ea59710c633cd90cb7))
- update links to match content in readme ([#386](https://github.com/uuidjs/uuid/issues/386)) ([44f2f86](https://github.com/uuidjs/uuid/commit/44f2f86e9d2bbf14ee5f0f00f72a3db1292666d4))
### [7.0.1](https://github.com/uuidjs/uuid/compare/v7.0.0...v7.0.1) (2020-02-25)
### Bug Fixes
- clean up esm builds for node and browser ([#383](https://github.com/uuidjs/uuid/issues/383)) ([59e6a49](https://github.com/uuidjs/uuid/commit/59e6a49e7ce7b3e8fb0f3ee52b9daae72af467dc))
- provide browser versions independent from module system ([#380](https://github.com/uuidjs/uuid/issues/380)) ([4344a22](https://github.com/uuidjs/uuid/commit/4344a22e7aed33be8627eeaaf05360f256a21753)), closes [#378](https://github.com/uuidjs/uuid/issues/378)
## [7.0.0](https://github.com/uuidjs/uuid/compare/v3.4.0...v7.0.0) (2020-02-24)
### ⚠ BREAKING CHANGES
- The default export, which used to be the v4() method but which was already discouraged in v3.x of this library, has been removed.
- Explicitly note that deep imports of the different uuid version functions are deprecated and no longer encouraged and that ECMAScript module named imports should be used instead. Emit a deprecation warning for people who deep-require the different algorithm variants.
- Remove builtin support for insecure random number generators in the browser. Users who want that will have to supply their own random number generator function.
- Remove support for generating v3 and v5 UUIDs in Node.js<4.x
- Convert code base to ECMAScript Modules (ESM) and release CommonJS build for node and ESM build for browser bundlers.
### Features
- add UMD build to npm package ([#357](https://github.com/uuidjs/uuid/issues/357)) ([4e75adf](https://github.com/uuidjs/uuid/commit/4e75adf435196f28e3fbbe0185d654b5ded7ca2c)), closes [#345](https://github.com/uuidjs/uuid/issues/345)
- add various es module and CommonJS examples ([b238510](https://github.com/uuidjs/uuid/commit/b238510bf352463521f74bab175a3af9b7a42555))
- ensure that docs are up-to-date in CI ([ee5e77d](https://github.com/uuidjs/uuid/commit/ee5e77db547474f5a8f23d6c857a6d399209986b))
- hybrid CommonJS & ECMAScript modules build ([a3f078f](https://github.com/uuidjs/uuid/commit/a3f078faa0baff69ab41aed08e041f8f9c8993d0))
- remove insecure fallback random number generator ([3a5842b](https://github.com/uuidjs/uuid/commit/3a5842b141a6e5de0ae338f391661e6b84b167c9)), closes [#173](https://github.com/uuidjs/uuid/issues/173)
- remove support for pre Node.js v4 Buffer API ([#356](https://github.com/uuidjs/uuid/issues/356)) ([b59b5c5](https://github.com/uuidjs/uuid/commit/b59b5c5ecad271c5453f1a156f011671f6d35627))
- rename repository to github:uuidjs/uuid ([#351](https://github.com/uuidjs/uuid/issues/351)) ([c37a518](https://github.com/uuidjs/uuid/commit/c37a518e367ac4b6d0aa62dba1bc6ce9e85020f7)), closes [#338](https://github.com/uuidjs/uuid/issues/338)
### Bug Fixes
- add deep-require proxies for local testing and adjust tests ([#365](https://github.com/uuidjs/uuid/issues/365)) ([7fedc79](https://github.com/uuidjs/uuid/commit/7fedc79ac8fda4bfd1c566c7f05ef4ac13b2db48))
- add note about removal of default export ([#372](https://github.com/uuidjs/uuid/issues/372)) ([12749b7](https://github.com/uuidjs/uuid/commit/12749b700eb49db8a9759fd306d8be05dbfbd58c)), closes [#370](https://github.com/uuidjs/uuid/issues/370)
- deprecated deep requiring of the different algorithm versions ([#361](https://github.com/uuidjs/uuid/issues/361)) ([c0bdf15](https://github.com/uuidjs/uuid/commit/c0bdf15e417639b1aeb0b247b2fb11f7a0a26b23))
## [3.4.0](https://github.com/uuidjs/uuid/compare/v3.3.3...v3.4.0) (2020-01-16)
### Features
- rename repository to github:uuidjs/uuid ([#351](https://github.com/uuidjs/uuid/issues/351)) ([e2d7314](https://github.com/uuidjs/uuid/commit/e2d7314)), closes [#338](https://github.com/uuidjs/uuid/issues/338)
## [3.3.3](https://github.com/uuidjs/uuid/compare/v3.3.2...v3.3.3) (2019-08-19)
### Bug Fixes
- no longer run ci tests on node v4
- upgrade dependencies
## [3.3.2](https://github.com/uuidjs/uuid/compare/v3.3.1...v3.3.2) (2018-06-28)
### Bug Fixes
- typo ([305d877](https://github.com/uuidjs/uuid/commit/305d877))
## [3.3.1](https://github.com/uuidjs/uuid/compare/v3.3.0...v3.3.1) (2018-06-28)
### Bug Fixes
- fix [#284](https://github.com/uuidjs/uuid/issues/284) by setting function name in try-catch ([f2a60f2](https://github.com/uuidjs/uuid/commit/f2a60f2))
# [3.3.0](https://github.com/uuidjs/uuid/compare/v3.2.1...v3.3.0) (2018-06-22)
### Bug Fixes
- assignment to readonly property to allow running in strict mode ([#270](https://github.com/uuidjs/uuid/issues/270)) ([d062fdc](https://github.com/uuidjs/uuid/commit/d062fdc))
- fix [#229](https://github.com/uuidjs/uuid/issues/229) ([c9684d4](https://github.com/uuidjs/uuid/commit/c9684d4))
- Get correct version of IE11 crypto ([#274](https://github.com/uuidjs/uuid/issues/274)) ([153d331](https://github.com/uuidjs/uuid/commit/153d331))
- mem issue when generating uuid ([#267](https://github.com/uuidjs/uuid/issues/267)) ([c47702c](https://github.com/uuidjs/uuid/commit/c47702c))
### Features
- enforce Conventional Commit style commit messages ([#282](https://github.com/uuidjs/uuid/issues/282)) ([cc9a182](https://github.com/uuidjs/uuid/commit/cc9a182))
## [3.2.1](https://github.com/uuidjs/uuid/compare/v3.2.0...v3.2.1) (2018-01-16)
### Bug Fixes
- use msCrypto if available. Fixes [#241](https://github.com/uuidjs/uuid/issues/241) ([#247](https://github.com/uuidjs/uuid/issues/247)) ([1fef18b](https://github.com/uuidjs/uuid/commit/1fef18b))
# [3.2.0](https://github.com/uuidjs/uuid/compare/v3.1.0...v3.2.0) (2018-01-16)
### Bug Fixes
- remove mistakenly added typescript dependency, rollback version (standard-version will auto-increment) ([09fa824](https://github.com/uuidjs/uuid/commit/09fa824))
- use msCrypto if available. Fixes [#241](https://github.com/uuidjs/uuid/issues/241) ([#247](https://github.com/uuidjs/uuid/issues/247)) ([1fef18b](https://github.com/uuidjs/uuid/commit/1fef18b))
### Features
- Add v3 Support ([#217](https://github.com/uuidjs/uuid/issues/217)) ([d94f726](https://github.com/uuidjs/uuid/commit/d94f726))
# [3.1.0](https://github.com/uuidjs/uuid/compare/v3.1.0...v3.0.1) (2017-06-17)
### Bug Fixes
- (fix) Add .npmignore file to exclude test/ and other non-essential files from packing. (#183)
- Fix typo (#178)
- Simple typo fix (#165)
### Features
- v5 support in CLI (#197)
- V5 support (#188)
# 3.0.1 (2016-11-28)
- split uuid versions into separate files
# 3.0.0 (2016-11-17)
- remove .parse and .unparse
# 2.0.0
- Removed uuid.BufferClass
# 1.4.0
- Improved module context detection
- Removed public RNG functions
# 1.3.2
- Improve tests and handling of v1() options (Issue #24)
- Expose RNG option to allow for perf testing with different generators
# 1.3.0
- Support for version 1 ids, thanks to [@ctavan](https://github.com/ctavan)!
- Support for node.js crypto API
- De-emphasizing performance in favor of a) cryptographic quality PRNGs where available and b) more manageable code

View File

@@ -0,0 +1,18 @@
# Contributing
Please feel free to file GitHub Issues or propose Pull Requests. We're always happy to discuss improvements to this library!
## Testing
```shell
npm test
```
## Releasing
Releases are supposed to be done from master, version bumping is automated through [`standard-version`](https://github.com/conventional-changelog/standard-version):
```shell
npm run release -- --dry-run # verify output manually
npm run release # follow the instructions from the output of this command
```

View File

@@ -0,0 +1,9 @@
The MIT License (MIT)
Copyright (c) 2010-2020 Robert Kieffer and other contributors
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -0,0 +1,505 @@
<!--
-- This file is auto-generated from README_js.md. Changes should be made there.
-->
# uuid [![CI](https://github.com/uuidjs/uuid/workflows/CI/badge.svg)](https://github.com/uuidjs/uuid/actions?query=workflow%3ACI) [![Browser](https://github.com/uuidjs/uuid/workflows/Browser/badge.svg)](https://github.com/uuidjs/uuid/actions?query=workflow%3ABrowser)
For the creation of [RFC4122](http://www.ietf.org/rfc/rfc4122.txt) UUIDs
- **Complete** - Support for RFC4122 version 1, 3, 4, and 5 UUIDs
- **Cross-platform** - Support for ...
- CommonJS, [ECMAScript Modules](#ecmascript-modules) and [CDN builds](#cdn-builds)
- Node 8, 10, 12, 14
- Chrome, Safari, Firefox, Edge, IE 11 browsers
- Webpack and rollup.js module bundlers
- [React Native / Expo](#react-native--expo)
- **Secure** - Cryptographically-strong random values
- **Small** - Zero-dependency, small footprint, plays nice with "tree shaking" packagers
- **CLI** - Includes the [`uuid` command line](#command-line) utility
**Upgrading from `uuid@3.x`?** Your code is probably okay, but check out [Upgrading From `uuid@3.x`](#upgrading-from-uuid3x) for details.
## Quickstart
To create a random UUID...
**1. Install**
```shell
npm install uuid
```
**2. Create a UUID** (ES6 module syntax)
```javascript
import { v4 as uuidv4 } from 'uuid';
uuidv4(); // ⇨ '9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d'
```
... or using CommonJS syntax:
```javascript
const { v4: uuidv4 } = require('uuid');
uuidv4(); // ⇨ '1b9d6bcd-bbfd-4b2d-9b5d-ab8dfbbd4bed'
```
For timestamp UUIDs, namespace UUIDs, and other options read on ...
## API Summary
| | | |
| --- | --- | --- |
| [`uuid.NIL`](#uuidnil) | The nil UUID string (all zeros) | New in `uuid@8.3` |
| [`uuid.parse()`](#uuidparsestr) | Convert UUID string to array of bytes | New in `uuid@8.3` |
| [`uuid.stringify()`](#uuidstringifyarr-offset) | Convert array of bytes to UUID string | New in `uuid@8.3` |
| [`uuid.v1()`](#uuidv1options-buffer-offset) | Create a version 1 (timestamp) UUID | |
| [`uuid.v3()`](#uuidv3name-namespace-buffer-offset) | Create a version 3 (namespace w/ MD5) UUID | |
| [`uuid.v4()`](#uuidv4options-buffer-offset) | Create a version 4 (random) UUID | |
| [`uuid.v5()`](#uuidv5name-namespace-buffer-offset) | Create a version 5 (namespace w/ SHA-1) UUID | |
| [`uuid.validate()`](#uuidvalidatestr) | Test a string to see if it is a valid UUID | New in `uuid@8.3` |
| [`uuid.version()`](#uuidversionstr) | Detect RFC version of a UUID | New in `uuid@8.3` |
## API
### uuid.NIL
The nil UUID string (all zeros).
Example:
```javascript
import { NIL as NIL_UUID } from 'uuid';
NIL_UUID; // ⇨ '00000000-0000-0000-0000-000000000000'
```
### uuid.parse(str)
Convert UUID string to array of bytes
| | |
| --------- | ---------------------------------------- |
| `str` | A valid UUID `String` |
| _returns_ | `Uint8Array[16]` |
| _throws_ | `TypeError` if `str` is not a valid UUID |
Note: Ordering of values in the byte arrays used by `parse()` and `stringify()` follows the left &Rarr; right order of hex-pairs in UUID strings. As shown in the example below.
Example:
```javascript
import { parse as uuidParse } from 'uuid';
// Parse a UUID
const bytes = uuidParse('6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b');
// Convert to hex strings to show byte order (for documentation purposes)
[...bytes].map((v) => v.toString(16).padStart(2, '0')); // ⇨
// [
// '6e', 'c0', 'bd', '7f',
// '11', 'c0', '43', 'da',
// '97', '5e', '2a', '8a',
// 'd9', 'eb', 'ae', '0b'
// ]
```
### uuid.stringify(arr[, offset])
Convert array of bytes to UUID string
| | |
| -------------- | ---------------------------------------------------------------------------- |
| `arr` | `Array`-like collection of 16 values (starting from `offset`) between 0-255. |
| [`offset` = 0] | `Number` Starting index in the Array |
| _returns_ | `String` |
| _throws_ | `TypeError` if a valid UUID string cannot be generated |
Note: Ordering of values in the byte arrays used by `parse()` and `stringify()` follows the left &Rarr; right order of hex-pairs in UUID strings. As shown in the example below.
Example:
```javascript
import { stringify as uuidStringify } from 'uuid';
const uuidBytes = [
0x6e,
0xc0,
0xbd,
0x7f,
0x11,
0xc0,
0x43,
0xda,
0x97,
0x5e,
0x2a,
0x8a,
0xd9,
0xeb,
0xae,
0x0b,
];
uuidStringify(uuidBytes); // ⇨ '6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b'
```
### uuid.v1([options[, buffer[, offset]]])
Create an RFC version 1 (timestamp) UUID
| | |
| --- | --- |
| [`options`] | `Object` with one or more of the following properties: |
| [`options.node` ] | RFC "node" field as an `Array[6]` of byte values (per 4.1.6) |
| [`options.clockseq`] | RFC "clock sequence" as a `Number` between 0 - 0x3fff |
| [`options.msecs`] | RFC "timestamp" field (`Number` of milliseconds, unix epoch) |
| [`options.nsecs`] | RFC "timestamp" field (`Number` of nanseconds to add to `msecs`, should be 0-10,000) |
| [`options.random`] | `Array` of 16 random bytes (0-255) |
| [`options.rng`] | Alternative to `options.random`, a `Function` that returns an `Array` of 16 random bytes (0-255) |
| [`buffer`] | `Array \| Buffer` If specified, uuid will be written here in byte-form, starting at `offset` |
| [`offset` = 0] | `Number` Index to start writing UUID bytes in `buffer` |
| _returns_ | UUID `String` if no `buffer` is specified, otherwise returns `buffer` |
| _throws_ | `Error` if more than 10M UUIDs/sec are requested |
Note: The default [node id](https://tools.ietf.org/html/rfc4122#section-4.1.6) (the last 12 digits in the UUID) is generated once, randomly, on process startup, and then remains unchanged for the duration of the process.
Note: `options.random` and `options.rng` are only meaningful on the very first call to `v1()`, where they may be passed to initialize the internal `node` and `clockseq` fields.
Example:
```javascript
import { v1 as uuidv1 } from 'uuid';
uuidv1(); // ⇨ '2c5ea4c0-4067-11e9-8bad-9b1deb4d3b7d'
```
Example using `options`:
```javascript
import { v1 as uuidv1 } from 'uuid';
const v1options = {
node: [0x01, 0x23, 0x45, 0x67, 0x89, 0xab],
clockseq: 0x1234,
msecs: new Date('2011-11-01').getTime(),
nsecs: 5678,
};
uuidv1(v1options); // ⇨ '710b962e-041c-11e1-9234-0123456789ab'
```
### uuid.v3(name, namespace[, buffer[, offset]])
Create an RFC version 3 (namespace w/ MD5) UUID
API is identical to `v5()`, but uses "v3" instead.
&#x26a0;&#xfe0f; Note: Per the RFC, "_If backward compatibility is not an issue, SHA-1 [Version 5] is preferred_."
### uuid.v4([options[, buffer[, offset]]])
Create an RFC version 4 (random) UUID
| | |
| --- | --- |
| [`options`] | `Object` with one or more of the following properties: |
| [`options.random`] | `Array` of 16 random bytes (0-255) |
| [`options.rng`] | Alternative to `options.random`, a `Function` that returns an `Array` of 16 random bytes (0-255) |
| [`buffer`] | `Array \| Buffer` If specified, uuid will be written here in byte-form, starting at `offset` |
| [`offset` = 0] | `Number` Index to start writing UUID bytes in `buffer` |
| _returns_ | UUID `String` if no `buffer` is specified, otherwise returns `buffer` |
Example:
```javascript
import { v4 as uuidv4 } from 'uuid';
uuidv4(); // ⇨ '1b9d6bcd-bbfd-4b2d-9b5d-ab8dfbbd4bed'
```
Example using predefined `random` values:
```javascript
import { v4 as uuidv4 } from 'uuid';
const v4options = {
random: [
0x10,
0x91,
0x56,
0xbe,
0xc4,
0xfb,
0xc1,
0xea,
0x71,
0xb4,
0xef,
0xe1,
0x67,
0x1c,
0x58,
0x36,
],
};
uuidv4(v4options); // ⇨ '109156be-c4fb-41ea-b1b4-efe1671c5836'
```
### uuid.v5(name, namespace[, buffer[, offset]])
Create an RFC version 5 (namespace w/ SHA-1) UUID
| | |
| --- | --- |
| `name` | `String \| Array` |
| `namespace` | `String \| Array[16]` Namespace UUID |
| [`buffer`] | `Array \| Buffer` If specified, uuid will be written here in byte-form, starting at `offset` |
| [`offset` = 0] | `Number` Index to start writing UUID bytes in `buffer` |
| _returns_ | UUID `String` if no `buffer` is specified, otherwise returns `buffer` |
Note: The RFC `DNS` and `URL` namespaces are available as `v5.DNS` and `v5.URL`.
Example with custom namespace:
```javascript
import { v5 as uuidv5 } from 'uuid';
// Define a custom namespace. Readers, create your own using something like
// https://www.uuidgenerator.net/
const MY_NAMESPACE = '1b671a64-40d5-491e-99b0-da01ff1f3341';
uuidv5('Hello, World!', MY_NAMESPACE); // ⇨ '630eb68f-e0fa-5ecc-887a-7c7a62614681'
```
Example with RFC `URL` namespace:
```javascript
import { v5 as uuidv5 } from 'uuid';
uuidv5('https://www.w3.org/', uuidv5.URL); // ⇨ 'c106a26a-21bb-5538-8bf2-57095d1976c1'
```
### uuid.validate(str)
Test a string to see if it is a valid UUID
| | |
| --------- | --------------------------------------------------- |
| `str` | `String` to validate |
| _returns_ | `true` if string is a valid UUID, `false` otherwise |
Example:
```javascript
import { validate as uuidValidate } from 'uuid';
uuidValidate('not a UUID'); // ⇨ false
uuidValidate('6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b'); // ⇨ true
```
Using `validate` and `version` together it is possible to do per-version validation, e.g. validate for only v4 UUIds.
```javascript
import { version as uuidVersion } from 'uuid';
import { validate as uuidValidate } from 'uuid';
function uuidValidateV4(uuid) {
return uuidValidate(uuid) && uuidVersion(uuid) === 4;
}
const v1Uuid = 'd9428888-122b-11e1-b85c-61cd3cbb3210';
const v4Uuid = '109156be-c4fb-41ea-b1b4-efe1671c5836';
uuidValidateV4(v4Uuid); // ⇨ true
uuidValidateV4(v1Uuid); // ⇨ false
```
### uuid.version(str)
Detect RFC version of a UUID
| | |
| --------- | ---------------------------------------- |
| `str` | A valid UUID `String` |
| _returns_ | `Number` The RFC version of the UUID |
| _throws_ | `TypeError` if `str` is not a valid UUID |
Example:
```javascript
import { version as uuidVersion } from 'uuid';
uuidVersion('45637ec4-c85f-11ea-87d0-0242ac130003'); // ⇨ 1
uuidVersion('6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b'); // ⇨ 4
```
## Command Line
UUIDs can be generated from the command line using `uuid`.
```shell
$ uuid
ddeb27fb-d9a0-4624-be4d-4615062daed4
```
The default is to generate version 4 UUIDS, however the other versions are supported. Type `uuid --help` for details:
```shell
$ uuid --help
Usage:
uuid
uuid v1
uuid v3 <name> <namespace uuid>
uuid v4
uuid v5 <name> <namespace uuid>
uuid --help
Note: <namespace uuid> may be "URL" or "DNS" to use the corresponding UUIDs
defined by RFC4122
```
## ECMAScript Modules
This library comes with [ECMAScript Modules](https://www.ecma-international.org/ecma-262/6.0/#sec-modules) (ESM) support for Node.js versions that support it ([example](./examples/node-esmodules/)) as well as bundlers like [rollup.js](https://rollupjs.org/guide/en/#tree-shaking) ([example](./examples/browser-rollup/)) and [webpack](https://webpack.js.org/guides/tree-shaking/) ([example](./examples/browser-webpack/)) (targeting both, Node.js and browser environments).
```javascript
import { v4 as uuidv4 } from 'uuid';
uuidv4(); // ⇨ '1b9d6bcd-bbfd-4b2d-9b5d-ab8dfbbd4bed'
```
To run the examples you must first create a dist build of this library in the module root:
```shell
npm run build
```
## CDN Builds
### ECMAScript Modules
To load this module directly into modern browsers that [support loading ECMAScript Modules](https://caniuse.com/#feat=es6-module) you can make use of [jspm](https://jspm.org/):
```html
<script type="module">
import { v4 as uuidv4 } from 'https://jspm.dev/uuid';
console.log(uuidv4()); // ⇨ '1b9d6bcd-bbfd-4b2d-9b5d-ab8dfbbd4bed'
</script>
```
### UMD
To load this module directly into older browsers you can use the [UMD (Universal Module Definition)](https://github.com/umdjs/umd) builds from any of the following CDNs:
**Using [UNPKG](https://unpkg.com/uuid@latest/dist/umd/)**:
```html
<script src="https://unpkg.com/uuid@latest/dist/umd/uuidv4.min.js"></script>
```
**Using [jsDelivr](https://cdn.jsdelivr.net/npm/uuid@latest/dist/umd/)**:
```html
<script src="https://cdn.jsdelivr.net/npm/uuid@latest/dist/umd/uuidv4.min.js"></script>
```
**Using [cdnjs](https://cdnjs.com/libraries/uuid)**:
```html
<script src="https://cdnjs.cloudflare.com/ajax/libs/uuid/8.1.0/uuidv4.min.js"></script>
```
These CDNs all provide the same [`uuidv4()`](#uuidv4options-buffer-offset) method:
```html
<script>
uuidv4(); // ⇨ '55af1e37-0734-46d8-b070-a1e42e4fc392'
</script>
```
Methods for the other algorithms ([`uuidv1()`](#uuidv1options-buffer-offset), [`uuidv3()`](#uuidv3name-namespace-buffer-offset) and [`uuidv5()`](#uuidv5name-namespace-buffer-offset)) are available from the files `uuidv1.min.js`, `uuidv3.min.js` and `uuidv5.min.js` respectively.
## "getRandomValues() not supported"
This error occurs in environments where the standard [`crypto.getRandomValues()`](https://developer.mozilla.org/en-US/docs/Web/API/Crypto/getRandomValues) API is not supported. This issue can be resolved by adding an appropriate polyfill:
### React Native / Expo
1. Install [`react-native-get-random-values`](https://github.com/LinusU/react-native-get-random-values#readme)
1. Import it _before_ `uuid`. Since `uuid` might also appear as a transitive dependency of some other imports it's safest to just import `react-native-get-random-values` as the very first thing in your entry point:
```javascript
import 'react-native-get-random-values';
import { v4 as uuidv4 } from 'uuid';
```
Note: If you are using Expo, you must be using at least `react-native-get-random-values@1.5.0` and `expo@39.0.0`.
### Web Workers / Service Workers (Edge <= 18)
[In Edge <= 18, Web Crypto is not supported in Web Workers or Service Workers](https://caniuse.com/#feat=cryptography) and we are not aware of a polyfill (let us know if you find one, please).
## Upgrading From `uuid@7.x`
### Only Named Exports Supported When Using with Node.js ESM
`uuid@7.x` did not come with native ECMAScript Module (ESM) support for Node.js. Importing it in Node.js ESM consequently imported the CommonJS source with a default export. This library now comes with true Node.js ESM support and only provides named exports.
Instead of doing:
```javascript
import uuid from 'uuid';
uuid.v4();
```
you will now have to use the named exports:
```javascript
import { v4 as uuidv4 } from 'uuid';
uuidv4();
```
### Deep Requires No Longer Supported
Deep requires like `require('uuid/v4')` [which have been deprecated in `uuid@7.x`](#deep-requires-now-deprecated) are no longer supported.
## Upgrading From `uuid@3.x`
"_Wait... what happened to `uuid@4.x` - `uuid@6.x`?!?_"
In order to avoid confusion with RFC [version 4](#uuidv4options-buffer-offset) and [version 5](#uuidv5name-namespace-buffer-offset) UUIDs, and a possible [version 6](http://gh.peabody.io/uuidv6/), releases 4 thru 6 of this module have been skipped.
### Deep Requires Now Deprecated
`uuid@3.x` encouraged the use of deep requires to minimize the bundle size of browser builds:
```javascript
const uuidv4 = require('uuid/v4'); // <== NOW DEPRECATED!
uuidv4();
```
As of `uuid@7.x` this library now provides ECMAScript modules builds, which allow packagers like Webpack and Rollup to do "tree-shaking" to remove dead code. Instead, use the `import` syntax:
```javascript
import { v4 as uuidv4 } from 'uuid';
uuidv4();
```
... or for CommonJS:
```javascript
const { v4: uuidv4 } = require('uuid');
uuidv4();
```
### Default Export Removed
`uuid@3.x` was exporting the Version 4 UUID method as a default export:
```javascript
const uuid = require('uuid'); // <== REMOVED!
```
This usage pattern was already discouraged in `uuid@3.x` and has been removed in `uuid@7.x`.
----
Markdown generated from [README_js.md](README_js.md) by [![RunMD Logo](http://i.imgur.com/h0FVyzU.png)](https://github.com/broofa/runmd)

View File

@@ -0,0 +1,2 @@
#!/usr/bin/env node
require('../uuid-bin');

View File

@@ -0,0 +1,9 @@
export { default as v1 } from './v1.js';
export { default as v3 } from './v3.js';
export { default as v4 } from './v4.js';
export { default as v5 } from './v5.js';
export { default as NIL } from './nil.js';
export { default as version } from './version.js';
export { default as validate } from './validate.js';
export { default as stringify } from './stringify.js';
export { default as parse } from './parse.js';

View File

@@ -0,0 +1,215 @@
/*
* Browser-compatible JavaScript MD5
*
* Modification of JavaScript MD5
* https://github.com/blueimp/JavaScript-MD5
*
* Copyright 2011, Sebastian Tschan
* https://blueimp.net
*
* Licensed under the MIT license:
* https://opensource.org/licenses/MIT
*
* Based on
* A JavaScript implementation of the RSA Data Security, Inc. MD5 Message
* Digest Algorithm, as defined in RFC 1321.
* Version 2.2 Copyright (C) Paul Johnston 1999 - 2009
* Other contributors: Greg Holt, Andrew Kepert, Ydnar, Lostinet
* Distributed under the BSD License
* See http://pajhome.org.uk/crypt/md5 for more info.
*/
function md5(bytes) {
if (typeof bytes === 'string') {
var msg = unescape(encodeURIComponent(bytes)); // UTF8 escape
bytes = new Uint8Array(msg.length);
for (var i = 0; i < msg.length; ++i) {
bytes[i] = msg.charCodeAt(i);
}
}
return md5ToHexEncodedArray(wordsToMd5(bytesToWords(bytes), bytes.length * 8));
}
/*
* Convert an array of little-endian words to an array of bytes
*/
function md5ToHexEncodedArray(input) {
var output = [];
var length32 = input.length * 32;
var hexTab = '0123456789abcdef';
for (var i = 0; i < length32; i += 8) {
var x = input[i >> 5] >>> i % 32 & 0xff;
var hex = parseInt(hexTab.charAt(x >>> 4 & 0x0f) + hexTab.charAt(x & 0x0f), 16);
output.push(hex);
}
return output;
}
/**
* Calculate output length with padding and bit length
*/
function getOutputLength(inputLength8) {
return (inputLength8 + 64 >>> 9 << 4) + 14 + 1;
}
/*
* Calculate the MD5 of an array of little-endian words, and a bit length.
*/
function wordsToMd5(x, len) {
/* append padding */
x[len >> 5] |= 0x80 << len % 32;
x[getOutputLength(len) - 1] = len;
var a = 1732584193;
var b = -271733879;
var c = -1732584194;
var d = 271733878;
for (var i = 0; i < x.length; i += 16) {
var olda = a;
var oldb = b;
var oldc = c;
var oldd = d;
a = md5ff(a, b, c, d, x[i], 7, -680876936);
d = md5ff(d, a, b, c, x[i + 1], 12, -389564586);
c = md5ff(c, d, a, b, x[i + 2], 17, 606105819);
b = md5ff(b, c, d, a, x[i + 3], 22, -1044525330);
a = md5ff(a, b, c, d, x[i + 4], 7, -176418897);
d = md5ff(d, a, b, c, x[i + 5], 12, 1200080426);
c = md5ff(c, d, a, b, x[i + 6], 17, -1473231341);
b = md5ff(b, c, d, a, x[i + 7], 22, -45705983);
a = md5ff(a, b, c, d, x[i + 8], 7, 1770035416);
d = md5ff(d, a, b, c, x[i + 9], 12, -1958414417);
c = md5ff(c, d, a, b, x[i + 10], 17, -42063);
b = md5ff(b, c, d, a, x[i + 11], 22, -1990404162);
a = md5ff(a, b, c, d, x[i + 12], 7, 1804603682);
d = md5ff(d, a, b, c, x[i + 13], 12, -40341101);
c = md5ff(c, d, a, b, x[i + 14], 17, -1502002290);
b = md5ff(b, c, d, a, x[i + 15], 22, 1236535329);
a = md5gg(a, b, c, d, x[i + 1], 5, -165796510);
d = md5gg(d, a, b, c, x[i + 6], 9, -1069501632);
c = md5gg(c, d, a, b, x[i + 11], 14, 643717713);
b = md5gg(b, c, d, a, x[i], 20, -373897302);
a = md5gg(a, b, c, d, x[i + 5], 5, -701558691);
d = md5gg(d, a, b, c, x[i + 10], 9, 38016083);
c = md5gg(c, d, a, b, x[i + 15], 14, -660478335);
b = md5gg(b, c, d, a, x[i + 4], 20, -405537848);
a = md5gg(a, b, c, d, x[i + 9], 5, 568446438);
d = md5gg(d, a, b, c, x[i + 14], 9, -1019803690);
c = md5gg(c, d, a, b, x[i + 3], 14, -187363961);
b = md5gg(b, c, d, a, x[i + 8], 20, 1163531501);
a = md5gg(a, b, c, d, x[i + 13], 5, -1444681467);
d = md5gg(d, a, b, c, x[i + 2], 9, -51403784);
c = md5gg(c, d, a, b, x[i + 7], 14, 1735328473);
b = md5gg(b, c, d, a, x[i + 12], 20, -1926607734);
a = md5hh(a, b, c, d, x[i + 5], 4, -378558);
d = md5hh(d, a, b, c, x[i + 8], 11, -2022574463);
c = md5hh(c, d, a, b, x[i + 11], 16, 1839030562);
b = md5hh(b, c, d, a, x[i + 14], 23, -35309556);
a = md5hh(a, b, c, d, x[i + 1], 4, -1530992060);
d = md5hh(d, a, b, c, x[i + 4], 11, 1272893353);
c = md5hh(c, d, a, b, x[i + 7], 16, -155497632);
b = md5hh(b, c, d, a, x[i + 10], 23, -1094730640);
a = md5hh(a, b, c, d, x[i + 13], 4, 681279174);
d = md5hh(d, a, b, c, x[i], 11, -358537222);
c = md5hh(c, d, a, b, x[i + 3], 16, -722521979);
b = md5hh(b, c, d, a, x[i + 6], 23, 76029189);
a = md5hh(a, b, c, d, x[i + 9], 4, -640364487);
d = md5hh(d, a, b, c, x[i + 12], 11, -421815835);
c = md5hh(c, d, a, b, x[i + 15], 16, 530742520);
b = md5hh(b, c, d, a, x[i + 2], 23, -995338651);
a = md5ii(a, b, c, d, x[i], 6, -198630844);
d = md5ii(d, a, b, c, x[i + 7], 10, 1126891415);
c = md5ii(c, d, a, b, x[i + 14], 15, -1416354905);
b = md5ii(b, c, d, a, x[i + 5], 21, -57434055);
a = md5ii(a, b, c, d, x[i + 12], 6, 1700485571);
d = md5ii(d, a, b, c, x[i + 3], 10, -1894986606);
c = md5ii(c, d, a, b, x[i + 10], 15, -1051523);
b = md5ii(b, c, d, a, x[i + 1], 21, -2054922799);
a = md5ii(a, b, c, d, x[i + 8], 6, 1873313359);
d = md5ii(d, a, b, c, x[i + 15], 10, -30611744);
c = md5ii(c, d, a, b, x[i + 6], 15, -1560198380);
b = md5ii(b, c, d, a, x[i + 13], 21, 1309151649);
a = md5ii(a, b, c, d, x[i + 4], 6, -145523070);
d = md5ii(d, a, b, c, x[i + 11], 10, -1120210379);
c = md5ii(c, d, a, b, x[i + 2], 15, 718787259);
b = md5ii(b, c, d, a, x[i + 9], 21, -343485551);
a = safeAdd(a, olda);
b = safeAdd(b, oldb);
c = safeAdd(c, oldc);
d = safeAdd(d, oldd);
}
return [a, b, c, d];
}
/*
* Convert an array bytes to an array of little-endian words
* Characters >255 have their high-byte silently ignored.
*/
function bytesToWords(input) {
if (input.length === 0) {
return [];
}
var length8 = input.length * 8;
var output = new Uint32Array(getOutputLength(length8));
for (var i = 0; i < length8; i += 8) {
output[i >> 5] |= (input[i / 8] & 0xff) << i % 32;
}
return output;
}
/*
* Add integers, wrapping at 2^32. This uses 16-bit operations internally
* to work around bugs in some JS interpreters.
*/
function safeAdd(x, y) {
var lsw = (x & 0xffff) + (y & 0xffff);
var msw = (x >> 16) + (y >> 16) + (lsw >> 16);
return msw << 16 | lsw & 0xffff;
}
/*
* Bitwise rotate a 32-bit number to the left.
*/
function bitRotateLeft(num, cnt) {
return num << cnt | num >>> 32 - cnt;
}
/*
* These functions implement the four basic operations the algorithm uses.
*/
function md5cmn(q, a, b, x, s, t) {
return safeAdd(bitRotateLeft(safeAdd(safeAdd(a, q), safeAdd(x, t)), s), b);
}
function md5ff(a, b, c, d, x, s, t) {
return md5cmn(b & c | ~b & d, a, b, x, s, t);
}
function md5gg(a, b, c, d, x, s, t) {
return md5cmn(b & d | c & ~d, a, b, x, s, t);
}
function md5hh(a, b, c, d, x, s, t) {
return md5cmn(b ^ c ^ d, a, b, x, s, t);
}
function md5ii(a, b, c, d, x, s, t) {
return md5cmn(c ^ (b | ~d), a, b, x, s, t);
}
export default md5;

View File

@@ -0,0 +1 @@
export default '00000000-0000-0000-0000-000000000000';

View File

@@ -0,0 +1,35 @@
import validate from './validate.js';
function parse(uuid) {
if (!validate(uuid)) {
throw TypeError('Invalid UUID');
}
var v;
var arr = new Uint8Array(16); // Parse ########-....-....-....-............
arr[0] = (v = parseInt(uuid.slice(0, 8), 16)) >>> 24;
arr[1] = v >>> 16 & 0xff;
arr[2] = v >>> 8 & 0xff;
arr[3] = v & 0xff; // Parse ........-####-....-....-............
arr[4] = (v = parseInt(uuid.slice(9, 13), 16)) >>> 8;
arr[5] = v & 0xff; // Parse ........-....-####-....-............
arr[6] = (v = parseInt(uuid.slice(14, 18), 16)) >>> 8;
arr[7] = v & 0xff; // Parse ........-....-....-####-............
arr[8] = (v = parseInt(uuid.slice(19, 23), 16)) >>> 8;
arr[9] = v & 0xff; // Parse ........-....-....-....-############
// (Use "/" to avoid 32-bit truncation when bit-shifting high-order bytes)
arr[10] = (v = parseInt(uuid.slice(24, 36), 16)) / 0x10000000000 & 0xff;
arr[11] = v / 0x100000000 & 0xff;
arr[12] = v >>> 24 & 0xff;
arr[13] = v >>> 16 & 0xff;
arr[14] = v >>> 8 & 0xff;
arr[15] = v & 0xff;
return arr;
}
export default parse;

View File

@@ -0,0 +1 @@
export default /^(?:[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}|00000000-0000-0000-0000-000000000000)$/i;

View File

@@ -0,0 +1,19 @@
// Unique ID creation requires a high quality random # generator. In the browser we therefore
// require the crypto API and do not support built-in fallback to lower quality random number
// generators (like Math.random()).
var getRandomValues;
var rnds8 = new Uint8Array(16);
export default function rng() {
// lazy load so that environments that need to polyfill have a chance to do so
if (!getRandomValues) {
// getRandomValues needs to be invoked in a context where "this" is a Crypto implementation. Also,
// find the complete implementation of crypto (msCrypto) on IE11.
getRandomValues = typeof crypto !== 'undefined' && crypto.getRandomValues && crypto.getRandomValues.bind(crypto) || typeof msCrypto !== 'undefined' && typeof msCrypto.getRandomValues === 'function' && msCrypto.getRandomValues.bind(msCrypto);
if (!getRandomValues) {
throw new Error('crypto.getRandomValues() not supported. See https://github.com/uuidjs/uuid#getrandomvalues-not-supported');
}
}
return getRandomValues(rnds8);
}

View File

@@ -0,0 +1,96 @@
// Adapted from Chris Veness' SHA1 code at
// http://www.movable-type.co.uk/scripts/sha1.html
function f(s, x, y, z) {
switch (s) {
case 0:
return x & y ^ ~x & z;
case 1:
return x ^ y ^ z;
case 2:
return x & y ^ x & z ^ y & z;
case 3:
return x ^ y ^ z;
}
}
function ROTL(x, n) {
return x << n | x >>> 32 - n;
}
function sha1(bytes) {
var K = [0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6];
var H = [0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476, 0xc3d2e1f0];
if (typeof bytes === 'string') {
var msg = unescape(encodeURIComponent(bytes)); // UTF8 escape
bytes = [];
for (var i = 0; i < msg.length; ++i) {
bytes.push(msg.charCodeAt(i));
}
} else if (!Array.isArray(bytes)) {
// Convert Array-like to Array
bytes = Array.prototype.slice.call(bytes);
}
bytes.push(0x80);
var l = bytes.length / 4 + 2;
var N = Math.ceil(l / 16);
var M = new Array(N);
for (var _i = 0; _i < N; ++_i) {
var arr = new Uint32Array(16);
for (var j = 0; j < 16; ++j) {
arr[j] = bytes[_i * 64 + j * 4] << 24 | bytes[_i * 64 + j * 4 + 1] << 16 | bytes[_i * 64 + j * 4 + 2] << 8 | bytes[_i * 64 + j * 4 + 3];
}
M[_i] = arr;
}
M[N - 1][14] = (bytes.length - 1) * 8 / Math.pow(2, 32);
M[N - 1][14] = Math.floor(M[N - 1][14]);
M[N - 1][15] = (bytes.length - 1) * 8 & 0xffffffff;
for (var _i2 = 0; _i2 < N; ++_i2) {
var W = new Uint32Array(80);
for (var t = 0; t < 16; ++t) {
W[t] = M[_i2][t];
}
for (var _t = 16; _t < 80; ++_t) {
W[_t] = ROTL(W[_t - 3] ^ W[_t - 8] ^ W[_t - 14] ^ W[_t - 16], 1);
}
var a = H[0];
var b = H[1];
var c = H[2];
var d = H[3];
var e = H[4];
for (var _t2 = 0; _t2 < 80; ++_t2) {
var s = Math.floor(_t2 / 20);
var T = ROTL(a, 5) + f(s, b, c, d) + e + K[s] + W[_t2] >>> 0;
e = d;
d = c;
c = ROTL(b, 30) >>> 0;
b = a;
a = T;
}
H[0] = H[0] + a >>> 0;
H[1] = H[1] + b >>> 0;
H[2] = H[2] + c >>> 0;
H[3] = H[3] + d >>> 0;
H[4] = H[4] + e >>> 0;
}
return [H[0] >> 24 & 0xff, H[0] >> 16 & 0xff, H[0] >> 8 & 0xff, H[0] & 0xff, H[1] >> 24 & 0xff, H[1] >> 16 & 0xff, H[1] >> 8 & 0xff, H[1] & 0xff, H[2] >> 24 & 0xff, H[2] >> 16 & 0xff, H[2] >> 8 & 0xff, H[2] & 0xff, H[3] >> 24 & 0xff, H[3] >> 16 & 0xff, H[3] >> 8 & 0xff, H[3] & 0xff, H[4] >> 24 & 0xff, H[4] >> 16 & 0xff, H[4] >> 8 & 0xff, H[4] & 0xff];
}
export default sha1;

View File

@@ -0,0 +1,30 @@
import validate from './validate.js';
/**
* Convert array of 16 byte values to UUID string format of the form:
* XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
*/
var byteToHex = [];
for (var i = 0; i < 256; ++i) {
byteToHex.push((i + 0x100).toString(16).substr(1));
}
function stringify(arr) {
var offset = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : 0;
// Note: Be careful editing this code! It's been tuned for performance
// and works in ways you may not expect. See https://github.com/uuidjs/uuid/pull/434
var uuid = (byteToHex[arr[offset + 0]] + byteToHex[arr[offset + 1]] + byteToHex[arr[offset + 2]] + byteToHex[arr[offset + 3]] + '-' + byteToHex[arr[offset + 4]] + byteToHex[arr[offset + 5]] + '-' + byteToHex[arr[offset + 6]] + byteToHex[arr[offset + 7]] + '-' + byteToHex[arr[offset + 8]] + byteToHex[arr[offset + 9]] + '-' + byteToHex[arr[offset + 10]] + byteToHex[arr[offset + 11]] + byteToHex[arr[offset + 12]] + byteToHex[arr[offset + 13]] + byteToHex[arr[offset + 14]] + byteToHex[arr[offset + 15]]).toLowerCase(); // Consistency check for valid UUID. If this throws, it's likely due to one
// of the following:
// - One or more input array values don't map to a hex octet (leading to
// "undefined" in the uuid)
// - Invalid input values for the RFC `version` or `variant` fields
if (!validate(uuid)) {
throw TypeError('Stringified UUID is invalid');
}
return uuid;
}
export default stringify;

View File

@@ -0,0 +1,95 @@
import rng from './rng.js';
import stringify from './stringify.js'; // **`v1()` - Generate time-based UUID**
//
// Inspired by https://github.com/LiosK/UUID.js
// and http://docs.python.org/library/uuid.html
var _nodeId;
var _clockseq; // Previous uuid creation time
var _lastMSecs = 0;
var _lastNSecs = 0; // See https://github.com/uuidjs/uuid for API details
function v1(options, buf, offset) {
var i = buf && offset || 0;
var b = buf || new Array(16);
options = options || {};
var node = options.node || _nodeId;
var clockseq = options.clockseq !== undefined ? options.clockseq : _clockseq; // node and clockseq need to be initialized to random values if they're not
// specified. We do this lazily to minimize issues related to insufficient
// system entropy. See #189
if (node == null || clockseq == null) {
var seedBytes = options.random || (options.rng || rng)();
if (node == null) {
// Per 4.5, create and 48-bit node id, (47 random bits + multicast bit = 1)
node = _nodeId = [seedBytes[0] | 0x01, seedBytes[1], seedBytes[2], seedBytes[3], seedBytes[4], seedBytes[5]];
}
if (clockseq == null) {
// Per 4.2.2, randomize (14 bit) clockseq
clockseq = _clockseq = (seedBytes[6] << 8 | seedBytes[7]) & 0x3fff;
}
} // UUID timestamps are 100 nano-second units since the Gregorian epoch,
// (1582-10-15 00:00). JSNumbers aren't precise enough for this, so
// time is handled internally as 'msecs' (integer milliseconds) and 'nsecs'
// (100-nanoseconds offset from msecs) since unix epoch, 1970-01-01 00:00.
var msecs = options.msecs !== undefined ? options.msecs : Date.now(); // Per 4.2.1.2, use count of uuid's generated during the current clock
// cycle to simulate higher resolution clock
var nsecs = options.nsecs !== undefined ? options.nsecs : _lastNSecs + 1; // Time since last uuid creation (in msecs)
var dt = msecs - _lastMSecs + (nsecs - _lastNSecs) / 10000; // Per 4.2.1.2, Bump clockseq on clock regression
if (dt < 0 && options.clockseq === undefined) {
clockseq = clockseq + 1 & 0x3fff;
} // Reset nsecs if clock regresses (new clockseq) or we've moved onto a new
// time interval
if ((dt < 0 || msecs > _lastMSecs) && options.nsecs === undefined) {
nsecs = 0;
} // Per 4.2.1.2 Throw error if too many uuids are requested
if (nsecs >= 10000) {
throw new Error("uuid.v1(): Can't create more than 10M uuids/sec");
}
_lastMSecs = msecs;
_lastNSecs = nsecs;
_clockseq = clockseq; // Per 4.1.4 - Convert from unix epoch to Gregorian epoch
msecs += 12219292800000; // `time_low`
var tl = ((msecs & 0xfffffff) * 10000 + nsecs) % 0x100000000;
b[i++] = tl >>> 24 & 0xff;
b[i++] = tl >>> 16 & 0xff;
b[i++] = tl >>> 8 & 0xff;
b[i++] = tl & 0xff; // `time_mid`
var tmh = msecs / 0x100000000 * 10000 & 0xfffffff;
b[i++] = tmh >>> 8 & 0xff;
b[i++] = tmh & 0xff; // `time_high_and_version`
b[i++] = tmh >>> 24 & 0xf | 0x10; // include version
b[i++] = tmh >>> 16 & 0xff; // `clock_seq_hi_and_reserved` (Per 4.2.2 - include variant)
b[i++] = clockseq >>> 8 | 0x80; // `clock_seq_low`
b[i++] = clockseq & 0xff; // `node`
for (var n = 0; n < 6; ++n) {
b[i + n] = node[n];
}
return buf || stringify(b);
}
export default v1;

View File

@@ -0,0 +1,4 @@
import v35 from './v35.js';
import md5 from './md5.js';
var v3 = v35('v3', 0x30, md5);
export default v3;

View File

@@ -0,0 +1,64 @@
import stringify from './stringify.js';
import parse from './parse.js';
function stringToBytes(str) {
str = unescape(encodeURIComponent(str)); // UTF8 escape
var bytes = [];
for (var i = 0; i < str.length; ++i) {
bytes.push(str.charCodeAt(i));
}
return bytes;
}
export var DNS = '6ba7b810-9dad-11d1-80b4-00c04fd430c8';
export var URL = '6ba7b811-9dad-11d1-80b4-00c04fd430c8';
export default function (name, version, hashfunc) {
function generateUUID(value, namespace, buf, offset) {
if (typeof value === 'string') {
value = stringToBytes(value);
}
if (typeof namespace === 'string') {
namespace = parse(namespace);
}
if (namespace.length !== 16) {
throw TypeError('Namespace must be array-like (16 iterable integer values, 0-255)');
} // Compute hash of namespace and value, Per 4.3
// Future: Use spread syntax when supported on all platforms, e.g. `bytes =
// hashfunc([...namespace, ... value])`
var bytes = new Uint8Array(16 + value.length);
bytes.set(namespace);
bytes.set(value, namespace.length);
bytes = hashfunc(bytes);
bytes[6] = bytes[6] & 0x0f | version;
bytes[8] = bytes[8] & 0x3f | 0x80;
if (buf) {
offset = offset || 0;
for (var i = 0; i < 16; ++i) {
buf[offset + i] = bytes[i];
}
return buf;
}
return stringify(bytes);
} // Function#name is not settable on some platforms (#270)
try {
generateUUID.name = name; // eslint-disable-next-line no-empty
} catch (err) {} // For CommonJS default export support
generateUUID.DNS = DNS;
generateUUID.URL = URL;
return generateUUID;
}

View File

@@ -0,0 +1,24 @@
import rng from './rng.js';
import stringify from './stringify.js';
function v4(options, buf, offset) {
options = options || {};
var rnds = options.random || (options.rng || rng)(); // Per 4.4, set bits for version and `clock_seq_hi_and_reserved`
rnds[6] = rnds[6] & 0x0f | 0x40;
rnds[8] = rnds[8] & 0x3f | 0x80; // Copy bytes to buffer, if provided
if (buf) {
offset = offset || 0;
for (var i = 0; i < 16; ++i) {
buf[offset + i] = rnds[i];
}
return buf;
}
return stringify(rnds);
}
export default v4;

View File

@@ -0,0 +1,4 @@
import v35 from './v35.js';
import sha1 from './sha1.js';
var v5 = v35('v5', 0x50, sha1);
export default v5;

Some files were not shown because too many files have changed in this diff Show More