{"id":138,"date":"2024-05-05T13:55:36","date_gmt":"2024-05-05T17:55:36","guid":{"rendered":"https:\/\/coding101.xyz\/?p=138"},"modified":"2024-05-05T15:28:49","modified_gmt":"2024-05-05T19:28:49","slug":"s3-access-typescript","status":"publish","type":"post","link":"https:\/\/coding101.xyz\/?p=138","title":{"rendered":"S3 Access Typescript"},"content":{"rendered":"\n<p>This is about how operations are performed using AWS JavaScript libraries to access AWS S3 or similar services. <\/p>\n\n\n\n<!--more-->\n\n\n\n<p>While S3 was launched long time ago by Amazon, nowadays almost all cloud providers have a similar service, allowing the use of existing code to access those services, with minimum configuration changes. <\/p>\n\n\n\n<p>The configuration changes (endpoint) are presented here <a href=\"https:\/\/coding101.xyz\/index.php\/2023\/11\/29\/bucket-storage-similar-with-s3-at-linode\/\" data-type=\"link\" data-id=\"https:\/\/coding101.xyz\/index.php\/2023\/11\/29\/bucket-storage-similar-with-s3-at-linode\/\">S3 Linode<\/a>. This article is how to use version v3 of aws-sdk in order to access those services<\/p>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<h2 class=\"wp-block-heading\">Dependency<\/h2>\n<\/div><\/div>\n\n\n\n<p>The dependency is this: <code>@aws-sdk\/client-s3<\/code><\/p>\n\n\n\n<p>Imports for our example<\/p>\n\n\n\n<!-- HTML generated using hilite.me --><div style=\"background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;\"><pre style=\"margin: 0; line-height: 125%\"><span style=\"color: #0000ff\">import<\/span> {\n    DeleteObjectCommand,\n    DeleteObjectCommandInput,\n    GetObjectCommand,\n    GetObjectCommandInput,\n    PutObjectCommand,\n    S3Client\n} from <span style=\"color: #a31515\">&#39;@aws-sdk\/client-s3&#39;<\/span>\n<span style=\"color: #0000ff\">import<\/span> fs from <span style=\"color: #a31515\">&#39;fs&#39;<\/span>\n<\/pre><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Client instance<\/h2>\n\n\n\n<p>An instance of the client is needed for any access, and here is how you create an instance of the client:<\/p>\n\n\n\n<!-- HTML generated using hilite.me --><div style=\"background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;\"><pre style=\"margin: 0; line-height: 125%\"><span style=\"color: #0000ff\">const<\/span> getClient = () : S3Client =&gt;{\n    <span style=\"color: #008000\">\/\/ Set your AWS credentials from environment variables<\/span>\n    <span style=\"color: #0000ff\">const<\/span> accessKeyId = process.env[<span style=\"color: #a31515\">&#39;key&#39;<\/span>]!!;\n    <span style=\"color: #0000ff\">const<\/span> secretAccessKey = process.env[<span style=\"color: #a31515\">&#39;secret&#39;<\/span>]!!;\n\n    <span style=\"color: #0000ff\">const<\/span> endpoint: <span style=\"color: #2b91af\">string<\/span> = <span style=\"color: #a31515\">&#39;https:\/\/us-east-1.linodeobjects.com&#39;<\/span>;\n    <span style=\"color: #0000ff\">const<\/span> region: <span style=\"color: #2b91af\">string<\/span> = <span style=\"color: #a31515\">&#39;us-east-1&#39;<\/span>;\n\n    <span style=\"color: #008000\">\/\/ Configure the S3 client<\/span>\n    <span style=\"color: #0000ff\">const<\/span> credentials = {secretAccessKey, accessKeyId}\n    <span style=\"color: #0000ff\">return<\/span> <span style=\"color: #0000ff\">new<\/span> S3Client({region, endpoint, credentials});\n}\n<\/pre><\/div>\n\n\n\n<p>Note that the credentials are taken from the environment. No environment means the implementation looks in the normal places where AWS configuration is found. Note the endpoint, this makes the difference between going to Amazon or to an alternate cloud provider. In our case it is Linode<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Uploading object<\/h2>\n\n\n\n<p>Don&#8217;t pay attention to the fs dependency: we have an example here of how to upload a file, however any stream of data can be processed the same<\/p>\n\n\n\n<!-- HTML generated using hilite.me --><div style=\"background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;\"><pre style=\"margin: 0; line-height: 125%\"><span style=\"color: #0000ff\">const<\/span> putObject = async () =&gt; {\n    <span style=\"color: #0000ff\">const<\/span> s3Client = getClient();\n    <span style=\"color: #0000ff\">const<\/span> bucketName = <span style=\"color: #a31515\">&#39;bucket-storage&#39;<\/span>;\n    <span style=\"color: #0000ff\">const<\/span> filePath = <span style=\"color: #a31515\">&#39;c:\\\\temp\\\\alfa.dat&#39;<\/span>; <span style=\"color: #008000\">\/\/ Change this to the path of your file<\/span>\n    <span style=\"color: #0000ff\">const<\/span> fileStream = fs.createReadStream(filePath);\n\n    <span style=\"color: #008000\">\/\/ Specify the parameters for the upload<\/span>\n    <span style=\"color: #0000ff\">const<\/span> uploadParams = {\n        Bucket: <span style=\"color: #2b91af\">bucketName<\/span>,\n        Key: <span style=\"color: #a31515\">&#39;alfa.dat&#39;<\/span>, <span style=\"color: #008000\">\/\/ Set the key (file name) under which you want to store the file in S3<\/span>\n        Body: <span style=\"color: #2b91af\">fileStream<\/span>,\n    };\n\n    <span style=\"color: #0000ff\">const<\/span> command = <span style=\"color: #0000ff\">new<\/span> PutObjectCommand(uploadParams);\n    <span style=\"color: #0000ff\">const<\/span> response = await s3Client.send(command);\n    console.log(response);\n}\n<\/pre><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Delete object<\/h2>\n\n\n\n<!-- HTML generated using hilite.me --><div style=\"background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;\"><pre style=\"margin: 0; line-height: 125%\"><span style=\"color: #0000ff\">const<\/span> deleteObject = async() =&gt; {\n    <span style=\"color: #0000ff\">const<\/span> s3Client = getClient();\n    <span style=\"color: #0000ff\">const<\/span> bucketName = <span style=\"color: #a31515\">&#39;bucket-storage&#39;<\/span>;\n\n    <span style=\"color: #008000\">\/\/ Specify the parameters for the upload<\/span>\n    <span style=\"color: #0000ff\">const<\/span> deleteObjectParam : <span style=\"color: #2b91af\">DeleteObjectCommandInput<\/span> = {\n        Bucket: <span style=\"color: #2b91af\">bucketName<\/span>,\n        Key: <span style=\"color: #a31515\">&#39;alfa.dat&#39;<\/span>\n    };\n\n    <span style=\"color: #0000ff\">const<\/span> command = <span style=\"color: #0000ff\">new<\/span> DeleteObjectCommand(deleteObjectParam);\n    <span style=\"color: #0000ff\">const<\/span> response = await s3Client.send(command);\n    console.log(response);\n}\n<\/pre><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Get object<\/h2>\n\n\n\n<p>Note that the object comes in chunks so it has to be processed asynchronously; this eliminates the need to get all the data, putting it in temporary storage and then take it from there<\/p>\n\n\n\n<!-- HTML generated using hilite.me --><div style=\"background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;\"><pre style=\"margin: 0; line-height: 125%\"><span style=\"color: #0000ff\">const<\/span> getObject = async () =&gt; {\n    <span style=\"color: #0000ff\">const<\/span> s3Client = getClient();\n    <span style=\"color: #0000ff\">const<\/span> bucketName = <span style=\"color: #a31515\">&#39;bucket-storage&#39;<\/span>;\n\n    <span style=\"color: #0000ff\">const<\/span> getObjectParam : <span style=\"color: #2b91af\">GetObjectCommandInput<\/span> = {\n        Bucket: <span style=\"color: #2b91af\">bucketName<\/span>,\n        Key: <span style=\"color: #a31515\">&#39;alfa.dat&#39;<\/span>\n    }\n\n    <span style=\"color: #0000ff\">const<\/span> command: <span style=\"color: #2b91af\">GetObjectCommand<\/span> = <span style=\"color: #0000ff\">new<\/span> GetObjectCommand(getObjectParam);\n    <span style=\"color: #0000ff\">const<\/span> response = await s3Client.send(command);\n    <span style=\"color: #0000ff\">const<\/span> body = response.Body!!;\n\n    <span style=\"color: #0000ff\">const<\/span> fileStream = fs.createWriteStream(<span style=\"color: #a31515\">&#39;c:\\\\temp\\\\info.dat&#39;<\/span>)\n    await processBody(body, (chunk) =&gt; {fileStream.write(chunk)});\n    fileStream.close();\n}\n\n<span style=\"color: #0000ff\">const<\/span> processBody = async (body: <span style=\"color: #2b91af\">any<\/span>, callback: (bytes: <span style=\"color: #2b91af\">Uint8Array<\/span>) =&gt; <span style=\"color: #0000ff\">void<\/span>) : Promise&lt;<span style=\"color: #0000ff\">void<\/span>&gt; =&gt; {\n    <span style=\"color: #0000ff\">return<\/span> <span style=\"color: #0000ff\">new<\/span> Promise((resolve, reject) =&gt; {\n        body.on(<span style=\"color: #a31515\">&#39;data&#39;<\/span>, (chunk: <span style=\"color: #2b91af\">Uint8Array<\/span>) =&gt; {\n            callback(chunk);\n        });\n\n        body.once(<span style=\"color: #a31515\">&#39;end&#39;<\/span>, () =&gt; {\n            resolve()\n        })\n\n        body.once(<span style=\"color: #a31515\">&#39;error&#39;<\/span>, (err : <span style=\"color: #2b91af\">any<\/span>) =&gt; {\n            reject(err)\n        })\n    });\n}\n<\/pre><\/div>\n\n\n\n<p>In this case, method processBody actually listens to the events that are triggered by the response.Body and assist in processing them asynchronously. <\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is about how operations are performed using AWS JavaScript libraries to access AWS S3 or similar services.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-138","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/coding101.xyz\/index.php?rest_route=\/wp\/v2\/posts\/138","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/coding101.xyz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/coding101.xyz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/coding101.xyz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/coding101.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=138"}],"version-history":[{"count":11,"href":"https:\/\/coding101.xyz\/index.php?rest_route=\/wp\/v2\/posts\/138\/revisions"}],"predecessor-version":[{"id":151,"href":"https:\/\/coding101.xyz\/index.php?rest_route=\/wp\/v2\/posts\/138\/revisions\/151"}],"wp:attachment":[{"href":"https:\/\/coding101.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=138"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/coding101.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=138"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/coding101.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=138"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}