Tag Archives: AWS

AWS: Choose the Right AWS Database Service

Posted on by
Reply

AWS Database Services

AWS has a variety of different database options for different use cases. Use the table below to get a quick look at the AWS database portfolio.

Database TypeUse CasesAWS Service
RelationalTraditional applications, ERP, CRM, e-commerceAmazon RDS, Amazon Aurora, Amazon Redshift
Key-valueHigh-traffic web apps, e-commerce systems, gaming applicationsAmazon DynamoDB
In-memoryCaching, session management, gaming leaderboards, geospatial applicationsAmazon ElastiCache for Memcached, Amazon ElastiCache for Redis
DocumentContent management, catalogs, user profilesAmazon DocumentDB (with MongoDB compatibility)
Wide columnHigh-scale industrial apps for equipment maintenance, fleet management, and route optimizationAmazon Keyspaces (for Apache Cassandra)
GraphFraud detection, social networking, recommendation enginesAmazon Neptune
Time seriesIoT applications, DevOps, industrial telemetryAmazon Timestream
LedgerSystems of record, supply chain, registrations, banking transactionsAmazon QLDB

Breaking Up Applications and Databases

As the industry changes, applications and databases change too. Today, with larger applications, you no longer see just one database supporting it. Instead, these applications are being broken into smaller services, each with their own purpose-built database supporting it.

This shift removes the idea of a one-size-fits-all database and replaces it with a complimentary database strategy. You can give each database the appropriate functionality, performance, and scale that the workload requires.

Resources:

AWS: Introduction to Amazon DynamoDB

Posted on by
Reply

What Is Amazon DynamoDB?

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. DynamoDB lets you offload the administrative burdens of operating and scaling a distributed database so that you don’t have to worry about hardware provisioning, setup and configuration, replication, software patching, or cluster scaling.

With DynamoDB, you can create database tables that can store and retrieve any amount of data and serve any level of request traffic. You can scale up or scale down your tables’ throughput capacity without downtime or performance degradation. You can use the AWS Management Console to monitor resource utilization and performance metrics.

DynamoDB automatically spreads the data and traffic for your tables over a sufficient number of servers to handle your throughput and storage requirements, while maintaining consistent and fast performance. All of your data is stored on solid-state disks (SSDs) and is automatically replicated across multiple Availability Zones in an AWS Region, providing built-in high availability and data durability.

Core Components of Amazon DynamoDB

In DynamoDB, tables, items, and attributes are the core components that you work with. A table is a collection of items, and each item is a collection of attributes. DynamoDB uses primary keys to uniquely identify each item in a table and secondary indexes to provide more querying flexibility.

The following are the basic DynamoDB components:

Tables – Similar to other database systems, DynamoDB stores data in tables. A table is a collection of data. For example, see the example table called People that you could use to store personal contact information about friends, family, or anyone else of interest. You could also have a Cars table to store information about vehicles that people drive.

Items – Each table contains zero or more items. An item is a group of attributes that is uniquely identifiable among all of the other items. In a People table, each item represents a person. For a Cars table, each item represents one vehicle. Items in DynamoDB are similar in many ways to rows, records, or tuples in other database systems. In DynamoDB, there is no limit to the number of items you can store in a table.

Attributes – Each item is composed of one or more attributes. An attribute is a fundamental data element, something that does not need to be broken down any further. For example, an item in a People table contains attributes called PersonID, LastName, FirstName, and so on. For a Department table, an item might have attributes such as DepartmentID, Name, Manager, and so on. Attributes in DynamoDB are similar in many ways to fields or columns in other database systems.

Security with Amazon DynamoDB

DynamoDB also offers encryption at rest, which eliminates the operational burden and complexity involved in protecting sensitive data. For more information, see DynamoDB Encryption at Rest.

Below you can find additional resources for learning about Amazon DynamoDB:

Linux: Using lsblk and smartctl to display hard disk overall-health self-assessment

Posted on by
Reply

root@debian01:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme0n1 259:0 0 476.9G 0 disk
├─nvme0n1p1 259:1 0 512M 0 part /boot/efi
├─nvme0n1p2 259:2 0 488M 0 part /boot
└─nvme0n1p3 259:3 0 476G 0 part
└─nvme0n1p3_crypt 254:0 0 475.9G 0 crypt
├─debian01–vg-root 254:1 0 23.3G 0 lvm /
├─debian01–vg-var 254:2 0 9.3G 0 lvm /var
├─debian01–vg-swap_1 254:3 0 976M 0 lvm
├─debian01–vg-tmp 254:4 0 1.9G 0 lvm /tmp
└─debian01–vg-home 254:5 0 440.5G 0 lvm /home

root@debian01:~# smartctl -a –test=long /dev/nvme0n1
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.1.0-18-amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, http://www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number: SAMSUNG MZ9LQ512HBLU-00B00
Serial Number: S7DANXMW102944
Firmware Version: FXM7601Q
PCI Vendor/Subsystem ID: 0x144d
IEEE OUI Identifier: 0x002538
Total NVM Capacity: 512,110,190,592 [512 GB]
Unallocated NVM Capacity: 0
Controller ID: 5
NVMe Version: 1.4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 512,110,190,592 [512 GB]
Namespace 1 Utilization: 61,558,759,424 [61.5 GB]
Namespace 1 Formatted LBA Size: 512
Namespace 1 IEEE EUI-64: 002538 d130ba314d
Local Time is: Mon Mar 18 11:42:24 2024 CST
Firmware Updates (0x16): 3 Slots, no Reset required
Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x1e): Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 83 Celsius
Critical Comp. Temp. Threshold: 85 Celsius
Namespace 1 Features (0x10): NP_Fields

Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 5.12W – – 0 0 0 0 0 0
1 + 3.59W – – 1 1 1 1 0 0
2 + 2.92W – – 2 2 2 2 0 500
3 – 0.0500W – – 3 3 3 3 210 1200
4 – 0.0050W – – 4 4 4 4 1000 9000

Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 51 Celsius
Available Spare: 100%
Available Spare Threshold: 50%
Percentage Used: 0%
Data Units Read: 181,599 [92.9 GB]
Data Units Written: 1,857,619 [951 GB]
Host Read Commands: 1,898,681
Host Write Commands: 48,222,637
Controller Busy Time: 238
Power Cycles: 75
Power On Hours: 52
Unsafe Shutdowns: 61
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Warning Comp. Temperature Time: 153
Critical Comp. Temperature Time: 3
Temperature Sensor 1: 51 Celsius
Thermal Temp. 1 Transition Count: 1236
Thermal Temp. 2 Transition Count: 1014
Thermal Temp. 1 Total Time: 2672
Thermal Temp. 2 Total Time: 12386

Error Information (NVMe Log 0x01, 16 of 64 entries)
No Errors Logged

root@debian01:~#

AWS: Data Management tools

Posted on by
Reply

Amazon Web Services (AWS) offers a variety of data management tools and services to help organizations collect, store, process, analyze, and visualize data. Some of the key data management tools and services provided by AWS include:

  1. Amazon S3 (Simple Storage Service): Amazon S3 is an object storage service that offers industry-leading scalability, data availability, security, and performance. It is commonly used for storing data for analytics, backup and recovery, archiving, and content distribution.
  2. Amazon RDS (Relational Database Service): Amazon RDS is a managed relational database service that supports several database engines, including MySQL, PostgreSQL, MariaDB, Oracle, and Microsoft SQL Server. It automates administrative tasks such as hardware provisioning, database setup, patching, and backups, allowing users to focus on their applications.
  3. Amazon Redshift: Amazon Redshift is a fully managed data warehouse service that makes it easy to analyze large datasets using SQL queries. It offers fast query performance by using columnar storage and parallel processing, making it suitable for analytics and business intelligence workloads.
  4. Amazon DynamoDB: Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. It is suitable for applications that require low-latency data access and flexible data models.
  5. Amazon Aurora: Amazon Aurora is a high-performance, fully managed relational database service that is compatible with MySQL and PostgreSQL. It offers performance and availability similar to commercial databases at a fraction of the cost.
  6. AWS Glue: AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy to prepare and load data for analytics. It automatically discovers and catalogs datasets, generates ETL code to transform data, and schedules and monitors ETL jobs.
  7. Amazon EMR (Elastic MapReduce): Amazon EMR is a managed big data platform that simplifies the processing of large datasets using popular distributed computing frameworks such as Apache Hadoop, Apache Spark, and Presto. It automatically provisions and scales compute resources based on workload demand.
  8. Amazon Kinesis: Amazon Kinesis is a platform for collecting, processing, and analyzing real-time streaming data at scale. It offers services such as Kinesis Data Streams for ingesting streaming data, Kinesis Data Firehose for loading data into data lakes and analytics services, and Kinesis Data Analytics for processing and analyzing streaming data with SQL.
  9. Amazon Elasticsearch Service: Amazon Elasticsearch Service is a managed service that makes it easy to deploy, operate, and scale Elasticsearch clusters in the AWS Cloud. It is commonly used for log and event data analysis, full-text search, and real-time application monitoring.

These are just a few examples of the data management tools and services available on AWS. Depending on specific requirements and use cases, organizations can choose the most appropriate AWS services to meet their needs.