https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/Recent content in Amazon S3 onHugoenhttps://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/overview/Mon, 01 Jan 0001 00:00:00 +0000https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/overview/<ol id="toc"></ol> <script> // JavaScript to generate the table of contents from H2 headings document.addEventListener("DOMContentLoaded", function () { //get all h2 headings within the 'main' element and generate a toc with links to them //excluding h2 heading 'Feedback' if it exists const toc = document.getElementById("toc"); const headings = document.querySelectorAll("main h2"); headings.forEach(heading => { if (heading.textContent === "Feedback") { return; // Skip the 'Feedback' heading } const li = document.createElement("li"); const a = document.createElement("a"); const id = heading.textContent.toLowerCase().replace(/\s+/g, '-'); heading.id = id; // Set the id for the heading a.href = `#${id}`; a.textContent = heading.textContent; li.appendChild(a); toc.appendChild(li); }); }); </script> <h2 id="solution-overview">Solution Overview</h2> <p>S3 Protector automatically protects files that are uploaded to an Amazon S3 source bucket and writes the processed result to an output bucket. Amazon S3 event notifications trigger the protector, and Cloud Protect API applies the configured Protegrity protection rules to the selected columns in each file.</p>https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/large-files/Mon, 01 Jan 0001 00:00:00 +0000https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/large-files/<h2 id="overview">Overview</h2> <p>S3 Protector processes files uploaded to Amazon S3 by invoking an AWS Lambda function for each <code>ObjectCreated</code> event. A single Lambda invocation is bounded by a hard 15-minute timeout and a maximum of 10 240 MB RAM. <strong>Large-file processing</strong> removes those ceilings by automatically splitting supported files into chunks, protecting each chunk in a separate parallel Processor Lambda, and reassembling the results into a single output file.</p> <p>Two complementary mechanisms handle large files:</p>https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/known_limitations/Mon, 01 Jan 0001 00:00:00 +0000https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/known_limitations/<h1 id="known-limitations">Known Limitations</h1> <p>Known product limitations:</p> <ul> <li>For <code>.json</code> files, supported tabular layouts are flat JSON arrays and JSON Lines. Deeply nested JSON structures are not supported.</li> <li>Legacy Excel format (.xls) is not supported.</li> <li>Excel files support is limited to single worksheet tab.</li> <li>Parallel file processing in chunks is available for CSV, Parquet, and JSON files. Excel files do not support parallel processing.</li> <li>Parquet timestamps are supported for protect/unprotect operations, but the output type of protected/unprotected timestamp will be converted to string.</li> <li>FPE is supported only for ASCII values.</li> <li>Only the protect and unprotect operations are supported. The reprotect operation is not currently supported.</li> <li>For processing large files in parallel chunks source and target buckets must be in the same region.</li> <li>Entire file is loaded into RAM for processing if file output format is different from input format.</li> <li>Single-row-group Parquet files cannot be processed in parallel chunks.</li> <li>AWS Lambda timeout of 15 minutes applies to each invocation.</li> </ul>https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/upgrading/Mon, 01 Jan 0001 00:00:00 +0000https://docs.protegrity.com/cloud-protect/4.1.0/docs/aws/s3/upgrading/<ol id="toc"></ol> <script> // JavaScript to generate the table of contents from H2 headings document.addEventListener("DOMContentLoaded", function () { //get all h2 headings within the 'main' element and generate a toc with links to them //excluding h2 heading 'Feedback' if it exists const toc = document.getElementById("toc"); const headings = document.querySelectorAll("main h2"); headings.forEach(heading => { if (heading.textContent === "Feedback") { return; // Skip the 'Feedback' heading } const li = document.createElement("li"); const a = document.createElement("a"); const id = heading.textContent.toLowerCase().replace(/\s+/g, '-'); heading.id = id; // Set the id for the heading a.href = `#${id}`; a.textContent = heading.textContent; li.appendChild(a); toc.appendChild(li); }); }); </script> <h2 id="upgrade-process-overview">Upgrade Process Overview</h2> <p>The diagram below illustrates upgrade steps:</p>