What is difference between data lake and data warehouse?

Jovan Stosic January 17, 2023

A data lake contains all an organization’s data in a raw, unstructured form, and can store the data indefinitely — for immediate or future use. A data warehouse contains structured data that has been cleaned and processed, ready for strategic analysis based on predefined business needs.

What is difference between data lake and data warehouse? was last modified: January 17th, 2023 by Jovan Stosic

Data lake – Wikipedia

Jovan Stosic January 17, 2023

A data lake is a system or repository of data stored in its natural/raw format, usually object blobs or files. A data lake is usually a single store of data including raw copies of source system data, sensor data, social data etc., and transformed data used for tasks such as reporting, visualization, advanced analytics and machine learning. A data lake can include structured data from relational databases (rows and columns), semi-structured data (CSV, logs, XML, JSON), unstructured data (emails, documents, PDFs) and binary data (images, audio, video). A data lake can be established “on premises” (within an organization’s data centers) or “in the cloud” (using cloud services from vendors such as Amazon, Microsoft, or Google).
Source: Data lake – Wikipedia

Data lake – Wikipedia was last modified: January 17th, 2023 by Jovan Stosic

dm-crypt

Jovan Stosic January 17, 2023

dm-crypt is a transparent block device encryption subsystem in Linux kernel versions 2.6 and later and in DragonFly BSD. It is part of the device mapper (dm) infrastructure, and uses cryptographic routines from the kernel’s Crypto API. Unlike its predecessor cryptoloop, dm-crypt was designed to support advanced modes of operation, such as XTS, LRW and ESSIV (see disk encryption theory for further information), in order to avoid watermarking attacks. In addition to that, dm-crypt addresses some reliability problems of cryptoloop.

dm-crypt is implemented as a device mapper target and may be stacked on top of other device mapper transformations. It can thus encrypt whole disks (including removable media), partitions, software RAID volumes, logical volumes, as well as files. It appears as a block device, which can be used to back file systems, swap or as an LVM physical volume.

Some Linux distributions support the use of dm-crypt on the root file system. These distributions use initrd to prompt the user to enter a passphrase at the console, or insert a smart card prior to the normal boot process.
Source: dm-crypt – Wikipedia

dm-crypt was last modified: January 17th, 2023 by Jovan Stosic

GitHub – lbernstone/miniz-esp32: This is a full copy of miniz, modified to use psram if available on an esp32

Jovan Stosic January 17, 2023

Miniz is a lossless, high performance data compression library in a single source file that implements the zlib (RFC 1950) and Deflate (RFC 1951) compressed data format specification standards. It supports the most commonly used functions exported by the zlib library, but is a completely independent implementation so zlib’s licensing requirements do not apply. Miniz also contains simple to use functions for writing .PNG format image files and reading/writing/appending .ZIP format archives. Miniz’s compression speed has been tuned to be comparable to zlib’s, and it also has a specialized real-time compressor function designed to compare well against fastlz/minilzo.

Source: GitHub – lbernstone/miniz-esp32: This is a full copy of miniz, modified to use psram if available on an esp32

GitHub – lbernstone/miniz-esp32: This is a full copy of miniz, modified to use psram if available on an esp32 was last modified: January 17th, 2023 by Jovan Stosic