What does an AI engineer do?

An AI engineer designs, builds, evaluates and operates software that uses machine-learning models or foundation models. The work can include APIs, data pipelines, retrieval, tool integration, model evaluation, security, deployment and monitoring.

How long does it take to become an AI engineer?

The timeline depends on prior programming experience, study time and project depth. A learner starting from zero may need roughly 12 to 18 months or longer at 10 to 15 hours per week. An experienced software developer may be able to build a credible transition portfolio in about 6 to 9 months. These are planning estimates, not job guarantees.

Do I need advanced mathematics to become an AI engineer?

Most application-focused AI engineering roles require practical understanding of vectors, probability, statistics, optimization and model evaluation rather than advanced mathematical research. Roles focused on model development or research require deeper mathematics.

What projects should an AI engineer put in a portfolio?

Build two or three complete projects that demonstrate a real problem, sound architecture, evaluation, security, deployment and clear documentation. A production-style RAG assistant and a permissioned tool-using workflow are strong examples.

RUSTCODE

Q: Which MySQL version should I use in 2027?

MySQL 8.0 reached end of life in April 2026. MySQL 8.4 LTS is a conservative choice for production, and 9.7 LTS is the newer long-term support line. The commands in this guide are standard SQL for these releases, and version-specific differences are noted where they matter. Run SELECT VERSION(); to see what your server uses.

Q: Do I need FLUSH PRIVILEGES after GRANT or CREATE USER?

No. MySQL applies account changes made with CREATE USER, GRANT, REVOKE and ALTER USER immediately. FLUSH PRIVILEGES is only needed if you edit the grant tables directly, which is not recommended.

Q: Why should I avoid GRANT ALL PRIVILEGES ON *.*?

It gives the account control over every database on the server, including system schemas. If the credentials leak or the application has a bug, the damage is much larger. Grant privileges on a single database or table instead.

Q: Can I rename a database in MySQL?

Not directly, because there is no RENAME DATABASE statement. Create a new database and restore a dump into it, or move each table with RENAME TABLE old_db.table_name TO new_db.table_name.

Q: Should I use BOOLEAN or TINYINT(1)?

BOOLEAN is an alias for TINYINT(1), so they behave the same. Use BOOLEAN for readability, and avoid display widths on other integer types because they are deprecated.

Q: How should I store user passwords?

Never store them in plain text. Hash them in your application with bcrypt or Argon2 and store the result in a VARCHAR(255) column. Database account passwords are a separate concern and belong in a secrets manager.

MySQL Commands Cheat Sheet for Developers

2026-11-06T14:44:00.389+05:30

MySQL Commands Cheat Sheet for Developers (8.4 & 9.7 LTS) | RustcodeWeb

MySQL Commands Cheat Sheet for Developers

Beginner to intermediateCheat sheetMySQL 8.4 and 9.7 LTS

The commands you use every day for logging in, managing users, working with databases and tables, querying data, and backing up safely, updated for current MySQL releases.

How to use this sheet

Scan the table of contents, jump to the task you need, and copy the example. Replace the sample names with your own, and read the notes under each group before you run anything destructive.

8.4 and 9.7Current LTS releases

13Topic sections

SQL + CLIQueries and terminal commands

SafeLeast privilege and backups

1. Before you begin: versions and conventions

Every example below uses the same sample names, so you can read the sheet top to bottom or jump to the command you need.

Release	Status in November 2026	What to do
MySQL 8.0	Reached end of life in April 2026.	Plan an upgrade to 8.4 LTS or 9.7 LTS.
MySQL 8.4 LTS	Long-term support release, supported for several more years.	A safe default for production systems.
MySQL 9.7 LTS	Newer long-term support line released in April 2026.	Test your application before migrating.
Innovation releases	Short-lived releases for new features.	Use for evaluation, not long-lived production.

Check exact patch versions and support dates on the official MySQL site before you plan an upgrade. Run SELECT VERSION(); to see what your server uses.

Sample names

Database shop, user app_user, tables customers and orders. Replace them with your own.

Two kinds of blocks

bash blocks run in your terminal. sql blocks run inside the mysql> prompt.

Safe by default

Examples use IF EXISTS, least privilege and transactions wherever they reduce risk.

Test first. Run destructive commands such as DROP, TRUNCATE, DELETE and REVOKE on a test copy, and take a backup before you touch production data.

2. Connect and use the MySQL client

Start here: log in, check who and where you are, and run commands from files.

Log in

Prompts for the password so it never lands in your shell history.

mysql -u root -p

Connect to a host, port and database

Use this for remote servers or when you want to start inside a database.

mysql -u app_user -p -h 127.0.0.1 -P 3306 shop

Run one statement without opening the prompt

Handy in scripts and quick checks.

mysql -u root -p -e "SHOW DATABASES;"

Check the server version and current user

Confirm what you are connected to before you change anything.

SELECT VERSION();
SELECT CURRENT_USER();
SELECT DATABASE();

Run a SQL file

From the shell, or from inside the client with source.

mysql -u root -p shop < script.sql

source /path/to/script.sql;

Exit

Any of these closes the client.

exit;

Handy client shortcuts

Shortcut	What it does
`\G`	Ends a statement and prints rows vertically, useful for wide tables.
`\s`	Shows connection and server status.
`\c`	Cancels the statement you are typing.
`help`	Lists client commands.
`\q`	Quits the client.

Avoid -pYourPassword. Passwords typed on the command line can appear in shell history and process lists. Leave the value out so MySQL prompts you.

3. Users, roles and privileges

Give each application its own account with only the access it needs. Avoid using root outside administration.

Create a user

IF NOT EXISTS makes the command safe to re-run.

CREATE USER IF NOT EXISTS 'app_user'@'localhost'
  IDENTIFIED BY 'Use-a-long-random-password';

Change a password

Use ALTER USER for password changes.

ALTER USER 'app_user'@'localhost' IDENTIFIED BY 'New-long-random-password';

List users

Needs access to the mysql system schema.

SELECT User, Host FROM mysql.user;

Show grants

Check what a user can do, or what you can do.

SHOW GRANTS FOR 'app_user'@'localhost';
SHOW GRANTS;

Grant least privilege on one database

The right default for most applications.

GRANT SELECT, INSERT, UPDATE, DELETE ON shop.* TO 'app_user'@'localhost';

Grant full control of one database

For a trusted owner or migration user. Still limited to a single database.

GRANT ALL PRIVILEGES ON shop.* TO 'app_user'@'localhost';

Remove specific privileges, or all of them

Revoke only what you need to take away.

REVOKE INSERT, UPDATE ON shop.* FROM 'app_user'@'localhost';
REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'app_user'@'localhost';

Drop a user

IF EXISTS avoids an error when the account is already gone.

DROP USER IF EXISTS 'app_user'@'localhost';

Use roles for shared permissions

Roles let you manage permissions for groups of users in one place.

CREATE ROLE IF NOT EXISTS 'report_reader';
GRANT SELECT ON shop.* TO 'report_reader';
GRANT 'report_reader' TO 'analyst'@'localhost';
SET DEFAULT ROLE 'report_reader' TO 'analyst'@'localhost';

Avoid GRANT ALL PRIVILEGES ON *.*. It gives an account control over every database, including system schemas. Reserve it for a small number of administrator accounts, and never use it for an application.

Good to know: FLUSH PRIVILEGES; is not needed after CREATE USER, GRANT or REVOKE. MySQL 8.4 disables the legacy mysql_native_password plugin by default and 9.0 removed it, so use the default caching_sha2_password and update old clients instead.

4. Database operations

Create, select and remove databases, with an explicit character set.

Show databases

Lists every database your account can see.

SHOW DATABASES;

Create a database

Specify utf8mb4 so emoji and all languages store correctly.

CREATE DATABASE IF NOT EXISTS shop
  CHARACTER SET utf8mb4
  COLLATE utf8mb4_0900_ai_ci;

Select a database and check which one is active

Run this before table commands, or you will see "No database selected".

USE shop;
SELECT DATABASE();

Inspect how a database was created

Shows its character set and collation.

SHOW CREATE DATABASE shop;

Delete a database

Permanent. Back up first.

DROP DATABASE IF EXISTS shop;

Renaming: MySQL has no RENAME DATABASE. Create the new database, then dump and restore into it, or move tables with RENAME TABLE old_db.t TO new_db.t;.

5. Table operations

A modern table definition, plus the commands you use to inspect and change it.

Show tables, columns and the full definition

DESCRIBE is the quick view and SHOW CREATE TABLE is the exact one.

SHOW TABLES;
DESCRIBE customers;
SHOW CREATE TABLE customers\G

Create a table

Uses a hash column instead of a plain password, timestamps that maintain themselves and an explicit unique key.

CREATE TABLE IF NOT EXISTS customers (
  id            BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  name          VARCHAR(100)    NOT NULL,
  email         VARCHAR(255)    NOT NULL,
  password_hash VARCHAR(255)    NOT NULL,
  is_student    BOOLEAN         NOT NULL DEFAULT FALSE,
  created_at    DATETIME        NOT NULL DEFAULT CURRENT_TIMESTAMP,
  updated_at    DATETIME        NOT NULL DEFAULT CURRENT_TIMESTAMP
                                ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (id),
  UNIQUE KEY uq_customers_email (email)
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4;

Change a table

Add, change, rename and remove columns with ALTER TABLE.

ALTER TABLE customers ADD COLUMN phone VARCHAR(20) NULL AFTER email;
ALTER TABLE customers MODIFY COLUMN name VARCHAR(150) NOT NULL;
ALTER TABLE customers RENAME COLUMN phone TO mobile;
ALTER TABLE customers DROP COLUMN mobile;

Rename a table

Renames atomically.

RENAME TABLE customers TO clients;

Empty or delete a table

TRUNCATE removes every row and resets AUTO_INCREMENT. Neither can be rolled back.

TRUNCATE TABLE customers;
DROP TABLE IF EXISTS customers;

Passwords: Never store plaintext passwords. Hash them in your application with bcrypt or Argon2 and store the result in a VARCHAR(255) column.

6. Insert, read, update and delete

The four everyday operations, with the safety habits that prevent bad edits.

Insert one row or many

Always name the columns so the statement survives schema changes.

INSERT INTO customers (name, email, password_hash, is_student)
VALUES ('Asha Rao', 'asha@example.com', '$2b$12$exampleHashOnly', TRUE);

INSERT INTO customers (name, email, password_hash)
VALUES ('Dev Patel', 'dev@example.com', '$2b$12$exampleHashOnly'),
       ('Mia Chen',  'mia@example.com', '$2b$12$exampleHashOnly');

Insert or update (upsert)

Updates the row when the unique key already exists. The row alias syntax replaces the deprecated VALUES() function.

INSERT INTO customers (name, email, password_hash)
VALUES ('Asha R. Rao', 'asha@example.com', '$2b$12$exampleHashOnly') AS new_row
ON DUPLICATE KEY UPDATE name = new_row.name;

Read rows

Select only the columns you need, sort them and limit the result.

SELECT id, name, email
FROM customers
WHERE is_student = TRUE
ORDER BY created_at DESC
LIMIT 10;

Update rows

Run the matching SELECT first to confirm what will change.

UPDATE customers
SET name = 'Asha R. Rao'
WHERE id = 1;

Delete rows

Keep the WHERE clause.

DELETE FROM customers WHERE id = 1;

Use a transaction for related changes

Either all statements apply or none do. Works with InnoDB tables.

START TRANSACTION;
UPDATE customers SET is_student = FALSE WHERE id = 2;
-- check the result, then:
COMMIT;   -- or ROLLBACK;

Turn on a safety net for your session

Blocks UPDATE and DELETE statements that have no key-based WHERE.

SET sql_safe_updates = 1;

Never run UPDATE or DELETE without WHERE. Without it, every row in the table changes. Write the WHERE first, then the rest of the statement.

7. Indexes, keys and constraints

Indexes make reads fast, and constraints keep your data valid.

Create a related table with a foreign key

Foreign keys keep orders tied to real customers.

CREATE TABLE IF NOT EXISTS orders (
  id          BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  customer_id BIGINT UNSIGNED NOT NULL,
  total       DECIMAL(10,2)   NOT NULL,
  ordered_at  DATETIME        NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (id),
  KEY idx_orders_customer (customer_id),
  CONSTRAINT fk_orders_customer
    FOREIGN KEY (customer_id) REFERENCES customers (id)
    ON DELETE CASCADE
) ENGINE = InnoDB;

Add, inspect and drop indexes

Index columns you filter, join and sort on.

CREATE INDEX idx_customers_name ON customers (name);
CREATE INDEX idx_customers_created ON customers (created_at);
SHOW INDEX FROM customers;
DROP INDEX idx_customers_name ON customers;

Check whether a query uses an index

EXPLAIN shows the plan, and EXPLAIN ANALYZE also runs the query and reports real timings.

EXPLAIN SELECT * FROM customers WHERE email = 'asha@example.com';
EXPLAIN ANALYZE SELECT * FROM orders WHERE customer_id = 1;

Tip: Indexes speed up reads but slow writes and use disk space. Add them for measured needs instead of indexing every column.

8. Filtering, grouping and joins

Patterns you will reuse in almost every project.

Filter with operators

Pattern matching, lists and ranges.

SELECT * FROM customers WHERE name LIKE 'A%';
SELECT * FROM customers WHERE id IN (1, 2, 3);
SELECT * FROM orders
WHERE ordered_at BETWEEN '2026-01-01' AND '2026-12-31';

Group and aggregate

HAVING filters groups after they are counted.

SELECT is_student, COUNT(*) AS total
FROM customers
GROUP BY is_student
HAVING COUNT(*) > 1;

Join tables

INNER JOIN returns only matching rows.

SELECT c.name, o.total, o.ordered_at
FROM customers AS c
INNER JOIN orders AS o ON o.customer_id = c.id
ORDER BY o.ordered_at DESC;

Keep rows that have no match

LEFT JOIN keeps every customer, even those without orders.

SELECT c.name, COUNT(o.id) AS order_count
FROM customers AS c
LEFT JOIN orders AS o ON o.customer_id = c.id
GROUP BY c.id, c.name;

Paginate results

Use a stable ORDER BY so pages do not overlap.

SELECT id, name FROM customers ORDER BY id LIMIT 20 OFFSET 40;

9. Backup, restore and export

Run these in your terminal, not in the mysql> prompt.

Back up one database

--single-transaction takes a consistent InnoDB snapshot without locking tables.

mysqldump -u root -p --single-transaction --routines --triggers --events shop > shop_backup.sql

Back up and compress

Adds the date to the file name.

mysqldump -u root -p --single-transaction shop | gzip > shop_$(date +%F).sql.gz

Back up every database, one table, or structure only

Pick the variant that matches your need.

mysqldump -u root -p --single-transaction --all-databases > all_databases.sql
mysqldump -u root -p --single-transaction shop customers > customers.sql
mysqldump -u root -p --no-data shop > shop_schema.sql

Restore a backup

Create the database first if it does not exist.

mysql -u root -p -e "CREATE DATABASE IF NOT EXISTS shop"
mysql -u root -p shop < shop_backup.sql
gunzip < shop_2026-11-06.sql.gz | mysql -u root -p shop

Test your restores. A backup you have never restored is only a guess. Restore into a scratch database regularly and check the data.

10. Inspect the server and find slow spots

Quick checks for when something feels slow or stuck.

See running queries and stop one

Find the Id of a stuck query, then kill it.

SHOW FULL PROCESSLIST;
KILL 123;

Check connections and limits

Useful when you see "Too many connections".

SHOW STATUS LIKE 'Threads_connected';
SHOW VARIABLES LIKE 'max_connections';

Check character sets and SQL mode

Confirms the defaults your sessions use.

SHOW VARIABLES LIKE 'character_set%';
SELECT @@sql_mode;

Find the largest tables

Sizes in megabytes for one schema.

SELECT TABLE_NAME AS table_name,
       ROUND((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024, 2) AS size_mb
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = 'shop'
ORDER BY size_mb DESC;

Refresh table statistics

Helps the optimizer choose good plans after large data changes.

ANALYZE TABLE customers;

11. Quick reference

The whole sheet on one screen. Bookmark this section.

Task	Command
Log in	`mysql -u root -p`
Show databases	`SHOW DATABASES;`
Select a database	`USE shop;`
Show tables	`SHOW TABLES;`
Describe a table	`DESCRIBE customers;`
Create a user	`CREATE USER IF NOT EXISTS 'app_user'@'localhost' IDENTIFIED BY '...';`
Grant on one database	`GRANT SELECT, INSERT ON shop.* TO 'app_user'@'localhost';`
Show grants	`SHOW GRANTS FOR 'app_user'@'localhost';`
Insert a row	`INSERT INTO customers (name, email, password_hash) VALUES (...);`
Update a row	`UPDATE customers SET name = 'New' WHERE id = 1;`
Delete a row	`DELETE FROM customers WHERE id = 1;`
Add an index	`CREATE INDEX idx_name ON customers (name);`
Explain a query	`EXPLAIN SELECT ...;`
Back up a database	`mysqldump -u root -p --single-transaction shop > shop.sql`
Restore a backup	`mysql -u root -p shop < shop.sql`
See running queries	`SHOW FULL PROCESSLIST;`

12. Common mistakes and a safety checklist

Most expensive database incidents come from a short list of habits.

Running UPDATE or DELETE without WHEREWrite the filter first and preview it with SELECT.
Using root for applicationsCreate a dedicated user per application with only the privileges it needs.
Granting privileges on *.*Grant on database.* or on specific tables instead.
Typing passwords in commandsLet -p prompt you, or use a protected option file.
Dropping or truncating without a backupThese statements cannot be rolled back. Dump the data first.
Storing plaintext passwordsStore a bcrypt or Argon2 hash created by your application.
Indexing every columnIndex what you filter, join and sort on, and confirm with EXPLAIN.

13. Official documentation

Prefer the manual for your exact version, and check the release notes before upgrading.

MySQL 8.4 Reference Manual

The official manual for the 8.4 LTS release.

GRANT Statement

Privileges, scopes and role syntax.

mysqldump

Every backup option explained.

Data Types

Choose the right column types.

EXPLAIN Statement

Read query plans and tune slow queries.

MySQL Release Support Dates

Track supported versions before upgrading.

Frequently asked questions

Which MySQL version should I use in 2027?

MySQL 8.0 reached end of life in April 2026. MySQL 8.4 LTS is a conservative choice for production, and 9.7 LTS is the newer long-term support line. The commands in this guide are standard SQL for these releases, and version-specific differences are noted where they matter. Run SELECT VERSION(); to see what your server uses.

Do I need FLUSH PRIVILEGES after GRANT or CREATE USER?

No. MySQL applies account changes made with CREATE USER, GRANT, REVOKE and ALTER USER immediately. FLUSH PRIVILEGES is only needed if you edit the grant tables directly, which is not recommended.

Why should I avoid GRANT ALL PRIVILEGES ON *.*?

It gives the account control over every database on the server, including system schemas. If the credentials leak or the application has a bug, the damage is much larger. Grant privileges on a single database or table instead.

Can I rename a database in MySQL?

Not directly, because there is no RENAME DATABASE statement. Create a new database and restore a dump into it, or move each table with RENAME TABLE old_db.table_name TO new_db.table_name.

Should I use BOOLEAN or TINYINT(1)?

BOOLEAN is an alias for TINYINT(1), so they behave the same. Use BOOLEAN for readability, and avoid display widths on other integer types because they are deprecated.

How should I store user passwords?

Never store them in plain text. Hash them in your application with bcrypt or Argon2 and store the result in a VARCHAR(255) column. Database account passwords are a separate concern and belong in a secrets manager.

Conclusion: your next steps

Knowing the commands is the easy part. Using them safely is what makes you dependable. Keep this sheet nearby, follow the safety habits and test every change on a copy first.

Key takeaways

Use MySQL 8.4 LTS or 9.7 LTS, because 8.0 is no longer supported.
Give every application its own user with least privilege, and avoid global grants.
Name columns in INSERT statements and always use WHERE with UPDATE and DELETE.
Use transactions for related changes and IF EXISTS or IF NOT EXISTS where it prevents errors.
Hash passwords in your application, and never store them in plain text.
Back up with mysqldump, then practice restoring.

Build the habit: read the statement, preview it with SELECT, run it inside a transaction when you can, and keep a backup you have actually restored.

About this cheat sheet: Examples target MySQL 8.4 and 9.7 and use sample names. Syntax and defaults can change between releases, so check the manual for your exact version before running commands on production data.

How to Become an AI Engineer in 2027

2026-11-03T18:31:56.157+05:30

AI Engineer Roadmap 2027: Skills, Projects, Timeline and Career Guide | RustcodeWeb

How to Become an AI Engineer in 2027

Beginner to intermediate12–18 month path10–15 hrs/weekCareer roadmap25 min read

A practical path from programming fundamentals to building, evaluating and deploying reliable AI products. Learn the concepts, build real systems and prove that they work.

Start with the right expectation

AI engineering is not simply writing prompts or connecting an application to a model API. It combines software engineering, data handling, model understanding, evaluation, security and operations. Your goal is to build a useful system that behaves predictably—not to collect the largest number of AI tools.

10Learning phases

2–3Portfolio-quality projects

8 mo.Capstone project growth plan

WeeklyBuild, test and review

1. What does an AI engineer actually do?

An AI engineer develops software that uses machine-learning models or foundation models to solve practical problems. The exact job varies by company: some roles focus on LLM applications, while others involve model training, inference infrastructure, data pipelines or classical machine learning.

Build applications

Integrate models into APIs, backend services and user-facing products. Handle authentication, data flow, errors and user experience.

Connect knowledge and tools

Use retrieval-augmented generation (RAG), search, structured outputs and controlled tool calling to ground model responses and perform tasks.

Make systems dependable

Evaluate quality, protect data, control permissions, monitor latency and cost, and respond safely when models or dependencies fail.

AI engineer vs. adjacent roles

Role	Typical emphasis	Useful foundation
AI application engineer	Product features built with existing models, APIs, RAG and tools.	Software engineering, APIs, evaluation and deployment.
Machine learning engineer	Training and serving models, data pipelines, experimentation and optimization.	ML, statistics, data engineering and production systems.
Research engineer / scientist	Implementing or investigating new model architectures, methods and experiments.	Deeper mathematics, research methods and model internals.
ML platform / inference engineer	Model serving, GPU utilization, throughput, reliability and infrastructure.	Distributed systems, systems programming and performance engineering.

Important: Job titles are not standardized. Read the responsibilities in a job description and select the learning path that matches the work—not just the title.

2. Choose your starting point

You do not need to begin at Phase 1 if you already have the skills. Answer the questions below honestly, because the quickest route is the one that fills your real gaps.

Start at the first question you cannot answer with a confident yes.

Your background	Where to start	Focus on	Planning estimate
Complete beginner	Phase 1	Python, Git, APIs and testing before any model work.	12–18 months
Working software developer	Phase 2, skim 3–4	LLM apps, RAG, evaluation, security and deployment (Phases 5–9).	6–9 months
Data analyst or data scientist	Phase 1 gaps, then 5	Backend engineering, APIs, Docker, testing and production habits.	6–9 months
CS student or graduate	Phase 3, then 5	Shipping projects, evaluation and deployment that coursework rarely covers.	6–12 months

Tip: Test yourself with the checkpoint at the end of each phase. If you can pass it without help, skim the phase and move on. If not, spend the time there.

3. The skill map: learn in layers

The sequence below is deliberate. Strong engineering fundamentals make it easier to debug model behavior, build reliable retrieval and operate applications after deployment.

Software engineering · Python · APIs · SQL · Testing

Data · Probability · Statistics · Classical ML

Neural networks · PyTorch · Transformers

LLM application development · Prompt and context design

RAG · Search · Tool calling · Agent workflows

Evaluation · Security · Deployment · Monitoring

Build upward. You can specialize later, but do not skip the foundations required by your target role.

Core capabilities to demonstrate

Capability	What you should be able to explain or do
Programming	Write maintainable Python, use types, handle exceptions, structure modules and debug unfamiliar code.
Backend engineering	Design REST APIs, validate inputs, use async operations appropriately and write automated tests.
Data and ML	Prepare datasets, avoid leakage, select metrics and interpret model performance.
LLM systems	Manage prompts, context, structured outputs, token limits, retries and model-provider differences.
Retrieval and tools	Build searchable knowledge systems and give models narrowly scoped, validated tools.
Production readiness	Measure quality, latency and cost; secure secrets and data; deploy and monitor the service.

4. The 10-phase AI engineering roadmap

Use the time ranges as planning guidance, not rigid deadlines. Open each phase to see what to learn, what to build and the checkpoint to pass before moving on.

1Software engineering foundationsSuggested duration: Months 1–2

AI applications are software systems. Learn to build ordinary applications before adding model calls.

Learn

Python: data types, functions, classes, modules, exceptions, files and virtual environments.
Git and GitHub: commits, branches, pull requests and code review.
Linux terminal, environment variables and basic shell scripting.
HTTP, REST, JSON, authentication and API status codes.
SQL fundamentals, joins, indexes and transactions.
Testing with pytest; logging, debugging and type hints.
Basic data structures, algorithms and Big-O reasoning.

Build

A command-line expense or inventory tracker using SQLite.
A Python client for a public API with timeouts and error handling.
A small FastAPI service with validation and tests.
A GitHub repository with setup instructions and a useful README.

Checkpoint: Build a small API-backed application from a clean repository, explain its request flow, write tests and debug a failed request without following a tutorial step by step.

2Math, data and statisticsSuggested duration: Months 2–3

Learn enough mathematics to understand model behavior and interpret experiments. The depth required depends on whether you target application engineering or model research.

Learn

Vectors, matrices, dot products and matrix multiplication.
Probability, conditional probability, distributions and Bayes' rule.
Mean, variance, standard deviation, sampling and confidence intervals.
Derivatives, gradients, loss functions and optimization intuition.
NumPy, pandas and data visualization.

Build

Explore and clean a real public dataset.
Implement linear regression with NumPy.
Visualize distributions, correlations and missing values.
Explain embeddings and vector similarity with a small numerical example.

Checkpoint: Explain train/validation/test splits, overfitting, gradient descent and why data leakage makes evaluation misleading.

3Classical machine learningSuggested duration: Months 3–4

Learn

Regression, classification, clustering and baseline models.
Feature engineering, preprocessing and pipelines.
Cross-validation, regularization and hyperparameter search.
Precision, recall, F1, ROC-AUC, confusion matrices and RMSE.
Class imbalance, threshold selection and model interpretability basics.
scikit-learn workflows.

Build

A spam or sentiment classifier.
A tabular prediction project with a documented baseline.
A model comparison report using the right metrics.
A reproducible training and inference script.

Checkpoint: Given a dataset and problem statement, select a sensible metric, build a baseline, avoid leakage and explain the limitations of your result.

4Deep learning and transformersSuggested duration: Months 4–6

Understand the concepts behind modern models. You do not need to train a frontier-scale model to become an effective AI application engineer.

Learn

Neural networks, activations, backpropagation and gradient descent.
PyTorch tensors, datasets, dataloaders and training loops.
Loss functions, optimizers, regularization and checkpoints.
Tokenization, embeddings, self-attention and transformer blocks.
Encoder-only, decoder-only and encoder-decoder model patterns.
Pretraining, inference, transfer learning and fine-tuning concepts.

Build

A small image or tabular classifier in PyTorch.
A text classification experiment using a pretrained model.
A tiny attention implementation or visualization.
A short technical note explaining next-token prediction.

Checkpoint: Explain tokens, embeddings, attention, context windows, inference and why a model's output is probabilistic.

5LLM application engineeringSuggested duration: Months 6–8

Move from notebooks to applications. Learn to treat model calls as unreliable external dependencies and design around that reality.

Learn

Model APIs, SDKs, streaming, tokenization and context limits.
System instructions, few-shot examples and prompt versioning.
Structured output, JSON Schema and Pydantic validation.
Timeouts, retries with backoff, rate limits and fallbacks.
Conversation state, context selection and summarization.
Model selection and routing based on quality, latency and cost.
Input/output logging with appropriate privacy controls.

Build

A streaming assistant with a clean backend API.
A document-to-validated-JSON extraction service.
A multi-provider model adapter behind one interface.
A summarization tool with input limits and failure handling.

Checkpoint: Your application handles malformed output, provider errors, rate limits and long inputs. It validates responses instead of trusting generated text blindly.

6Retrieval-augmented generation and searchSuggested duration: Months 8–10

RAG helps an application retrieve relevant external or private information at query time. Retrieval quality and source handling are as important as the language model.

Conceptual RAG flow. Real systems also need document updates, access control, filtering and evaluation.

Learn

Document parsing, metadata and chunking trade-offs.
Embedding models and similarity search.
Vector search and keyword search; hybrid retrieval.
Filtering, reranking, query rewriting and context assembly.
Source attribution, citations and access-control filtering.
Retrieval metrics such as Recall@k and MRR.
RAG failure modes: missing, stale or conflicting evidence.

Build

A PDF question-answering app with page-level citations.
A searchable documentation assistant.
A knowledge base with metadata filters and document updates.
A small test set to compare chunking and retrieval strategies.

Checkpoint: The application retrieves relevant passages, cites the evidence, respects document permissions and can say when its sources do not support an answer.

7Tool calling and agent workflowsSuggested duration: Months 10–12

Agents combine model reasoning with tools and state. Start with deterministic workflows and introduce autonomy only when it adds measurable value.

Learn

Function calling and schema-validated tool arguments.
Tool execution, result handling and bounded agent loops.
Workflow orchestration, state and durable execution basics.
Model Context Protocol (MCP) concepts and tool interoperability.
Human approval, scoped permissions and audit logs.
Prompt injection, untrusted tool output and sandboxing.

Build

A research assistant that produces a source-linked report.
An agent that reads data but cannot modify it.
A workflow that drafts an action and requires approval before execution.
A tool-using support assistant with explicit permission boundaries.

Checkpoint: Every tool has a clear schema and permission boundary. The workflow handles tool errors, limits repeated actions and requests approval for sensitive or irreversible operations.

8Evaluation, safety and observabilitySuggested duration: Months 12–14

Evaluation is not a final polish step. It is how you decide whether the system is useful and whether a change made it better or worse.

Learn

Task-specific test cases and human-reviewed reference answers.
Quality dimensions: correctness, relevance, completeness and groundedness.
RAG evaluation: retrieval recall, ranking quality and citation support.
Agent evaluation: task completion, tool selection and safe execution.
Latency percentiles, throughput, token usage and cost per task.
Tracing, structured logs, monitoring and alerting.
Privacy, data retention, secrets and prompt-injection defenses.

Build

A 50–100-question evaluation set for your RAG system.
Automated regression tests for prompts and retrieval changes.
A dashboard for latency, errors, token usage and estimated cost.
Red-team cases for prompt injection and unauthorized data access.
A review process for failures and user feedback.

Checkpoint: You can show measured quality and operational performance, explain known failure modes and compare a new version against a repeatable baseline.

9Deployment and production operationsSuggested duration: Months 14–16

Learn

FastAPI, Docker and one cloud platform (AWS, Azure or GCP).
PostgreSQL, Redis, queues and background workers.
Authentication, authorization, rate limiting and secrets management.
CI/CD, environment configuration and database migrations.
Logging, tracing, dashboards, alerts and incident basics.
Local inference concepts; model serving, batching and quantization at an introductory level.
Backups, rollback and cost controls.

Build

Deploy your RAG application as a containerized service.
Add authentication, per-user access controls and rate limits.
Set up CI tests and a repeatable deployment process.
Publish a demo with monitoring and an architecture diagram.

Checkpoint: Another developer can run your application from the README. The deployed service has secure configuration, tests, monitoring and a documented recovery path.

10Portfolio, interviews and specializationSuggested duration: Months 16–18 and ongoing

Build a focused portfolio

One production-style RAG application with citations and evaluation.
One permissioned tool-using workflow with human approval.
One ML or deep-learning project if your target roles require model development.

Every case study should explain

The user problem, requirements and why AI is appropriate.
Architecture, data flow and technology choices.
Evaluation method, measured results and limitations.
Security, privacy, latency and cost considerations.
What failed, what you changed and what you would improve next.

Prepare for interviews

Python, data structures, algorithms, SQL and API design.
ML fundamentals, model evaluation and data leakage.
RAG architecture, retrieval trade-offs and evaluation.
Agent design, tool permissions, prompt injection and failure handling.
System design, deployment and production debugging.

Checkpoint: You can demo your work, defend design decisions with evidence and explain trade-offs without relying on framework-specific terminology.

5. Portfolio projects that demonstrate real ability

A strong project is not defined by how many frameworks it uses. It is defined by a clear problem, sound engineering, measurable results and thoughtful handling of failure.

Project A: Knowledge-base assistant

Problem: Help a team find answers across technical documents, manuals or policies.

Document ingestion, chunking and metadata.
Hybrid retrieval, reranking and citations.
Access control and a refusal path when evidence is missing.
Evaluation set with retrieval and answer-quality metrics.

Demonstrates: RAG, backend engineering, evaluation and data security.

Project B: Controlled operations agent

Problem: Let users request actions through a natural-language interface without giving the model unrestricted access.

Typed tools with strict input validation.
Read-only operations by default.
Human confirmation for writes or irreversible actions.
Audit trail, retries and tool-failure handling.

Demonstrates: Tool use, workflow design, safety and reliability.

Project C: Document intelligence API

Problem: Extract consistent structured data from invoices, forms or technical reports.

File validation and extraction pipeline.
Schema-constrained model output.
Validation rules and human review for low-confidence cases.
Batch processing, logs and cost reporting.

Demonstrates: LLM integration, structured output and production APIs.

Project D: Small ML prediction service

Problem: Train and serve a model for a well-defined classification or regression task.

Reproducible data preparation and baseline.
Appropriate metrics and error analysis.
Versioned model artifact and inference API.
Monitoring plan for data and performance drift.

Demonstrates: Classical ML, model lifecycle and deployment fundamentals.

Portfolio quality bar: Include a working demo or short walkthrough, architecture diagram, setup instructions, sample data, tests, evaluation results, known limitations and a clear explanation of your own contribution.

6. Grow one capstone project over eight months

Rather than starting a new tutorial every week, evolve one useful application: a Company Knowledge Copilot. Each month should add a capability and improve the quality of the existing system.

Build stage	Deliverable	Evidence to capture
Month 1	Python prototype that searches one document collection.	Clear problem statement and baseline examples.
Month 2	FastAPI backend, request validation and streaming.	API tests and error-handling examples.
Month 3	Document ingestion, chunking and embedding pipeline.	Ingestion tests and indexing documentation.
Month 4	Retrieval, reranking and source citations.	Retrieval comparison and citation examples.
Month 5	Evaluation suite, feedback capture and guardrails.	Baseline metrics and regression results.
Month 6	One or two carefully scoped tools.	Permission tests and approval flow.
Month 7	Containerized deployment and monitoring.	Deployment guide, logs and latency/cost report.
Month 8	Polished demo, case study and technical walkthrough.	Public portfolio page and lessons learned.

The eight-month project plan can overlap with the learning phases. It is not an additional eight months after the full roadmap.

7. A realistic weekly study plan

At 10–15 hours per week, protect time for implementation and debugging. Watching courses is useful, but it should not dominate your schedule.

Session	Focus	Suggested time	Output
Session 1	Learn one concept and take concise notes.	2 hours	Concept summary and small example.
Session 2	Hands-on coding exercises.	2 hours	Working implementation.
Session 3	Capstone development.	2–3 hours	One feature or measurable improvement.
Session 4	Read documentation and inspect a reference implementation.	1–2 hours	Notes on design choices and trade-offs.
Session 5	Testing, debugging and evaluation.	2–3 hours	Tests, benchmark or failure analysis.
Session 6	Review and portfolio maintenance.	1–2 hours	Updated README, issue list and next steps.

The loop to repeat on every topic

Reading alone does not build skill. Use the same four-step loop for each concept, then repeat it with a harder version of the same project.

Documenting what failed is what turns practice into portfolio evidence.

Your first 30 days

Week	Goal	Deliverable
Week 1	Set up Python 3.11+, Git, an editor, a virtual environment and a GitHub account.	A repository with a script, one passing `pytest` test and a README.
Week 2	Learn functions, classes, files, exceptions and SQLite.	A command-line tracker that stores data and handles bad input.
Week 3	Learn HTTP, JSON and FastAPI basics.	A small API with request validation and two tested endpoints.
Week 4	Make your first model call from code, with a timeout and logging.	An endpoint that returns validated JSON and logs latency and token usage.

Week 1 setup commands

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install fastapi "uvicorn[standard]" pydantic pytest
pytest

Consistency beats an unrealistic schedule. If you only have five hours per week, keep the same order and extend the timeline. Do not skip testing and project work to meet an arbitrary deadline.

8. Reference architecture of a production AI application

Most AI products share the same shape. Knowing the parts, and how each one fails, makes your projects look like real systems instead of demos.

Every request follows one path, so each step can be logged, limited and tested.

Component	Responsibility	Failure to test for
API layer	Authenticate users, validate input and enforce rate limits.	Oversized inputs, missing auth and request floods.
Orchestrator	Build context, call the model, run tools and bound the loop.	Infinite loops, context overflow and wrong tool choice.
Model provider	Generate text, structured output and embeddings.	Timeouts, rate limits, malformed output and model changes.
Retrieval store	Return relevant, permitted passages with metadata.	Stale documents, poor chunking and leaked private data.
Tools and data	Perform narrow, validated actions with least privilege.	Unsafe arguments, duplicate actions and prompt injection.
Observability	Record traces, quality scores, latency and cost.	Silent quality regressions and cost spikes.

Starter pattern: never trust raw model output

Validate every response against a schema and retry a limited number of times. This short pattern is the backbone of reliable extraction, routing and tool-calling code.

from pydantic import BaseModel, ValidationError

class Invoice(BaseModel):
    vendor: str
    total: float
    currency: str

def extract_invoice(text: str, max_attempts: int = 3) -> Invoice:
    last_error = None
    for attempt in range(max_attempts):
        raw = call_model(  # your wrapper: timeout, logging, cost tracking
            prompt=f"Return JSON with vendor, total, currency.\n\n{text}",
            timeout=30,
        )
        try:
            return Invoice.model_validate_json(raw)
        except ValidationError as err:
            last_error = err  # log the failure, then retry
    raise RuntimeError("Invalid model output") from last_error

call_model is a placeholder for your own provider wrapper. Many SDKs also offer built-in structured outputs. Keep the validation step anyway, because it protects you when a provider, model or prompt changes.

9. Tools and technologies to learn

Choose one tool for each job first. Learn the underlying concept before adding another framework. Tool popularity changes; engineering fundamentals last longer.

Programming and data

Backend and application development

LLM and retrieval stack

Evaluation and operations

Suggested order: Start with Python, Git, SQL, FastAPI and pytest. Add one model SDK, then build retrieval with PostgreSQL/pgvector or a vector database. Introduce orchestration frameworks only when they solve a problem your plain implementation cannot handle cleanly.

10. Key terms in plain English

Interviewers expect you to explain these clearly, without jargon.

Term	Meaning	Why it matters
Token	A chunk of text, often part of a word, that a model reads and writes.	Tokens drive cost, speed and context limits.
Embedding	A list of numbers that represents meaning, so similar content sits close together.	It powers semantic search and retrieval.
Context window	The maximum text a model can consider in one request.	It limits how much evidence and history you can include.
Hallucination	A fluent answer that is unsupported or false.	It is why you need grounding, citations and evaluation.
RAG	Retrieving relevant documents at query time and giving them to the model.	It adds current or private knowledge without retraining.
Reranker	A model that reorders retrieved passages by relevance.	It often improves answer quality cheaply.
Tool calling	The model asks your code to run a function with structured arguments.	It lets an application act, so permissions matter.
MCP	An open protocol for connecting AI applications to tools and data sources.	It reduces custom integration work.
Eval set	A fixed set of test questions with expected results.	It shows whether a change helped or hurt.
Prompt injection	Untrusted text that tries to override your instructions.	It is a core security risk for retrieval and agents.
LoRA	A lightweight fine-tuning method that trains a small set of extra weights.	It makes adaptation cheaper than full fine-tuning.
p95 latency	The response time that 95 percent of requests beat.	It reflects what slow users actually experience.

11. Common mistakes that slow down learners

Starting with autonomous agentsBuild reliable APIs, structured outputs and retrieval before adding complex agent loops.
Collecting certificates instead of evidenceUse courses to learn, then demonstrate the skill in a tested and documented project.
Staying in notebooksMove at least one project into a maintainable backend with tests and deployment.
Skipping evaluationWithout a baseline and repeatable tests, you cannot know whether a change improved the system.
Fine-tuning too earlyFirst test whether better instructions, retrieval, context or a different model solves the problem.
Learning every frameworkUnderstand the architecture and trade-offs; frameworks are replaceable implementation choices.
Ignoring security and privacyDo not expose secrets, private documents or privileged tools to an unrestricted model context.
Overclaiming project resultsReport what you measured, how you measured it and where the system still fails.

12. Suggested 18-month timeline

This is one possible sequence for someone starting with limited programming experience and studying around 10–15 hours each week. People with prior software experience can compress the early phases.

Period	Primary focus	Milestone
Months 1–2	Python, Git, APIs, SQL and testing.	Build and test a small backend application.
Months 2–4	Math, data and classical ML.	Train, compare and explain a baseline model.
Months 4–6	Deep learning and transformer concepts.	Complete a small PyTorch experiment.
Months 6–8	LLM application engineering.	Ship a validated, error-tolerant model-backed API.
Months 8–10	RAG and semantic search.	Build a source-grounded knowledge assistant.
Months 10–12	Tool calling and agent workflows.	Build a permissioned workflow with approval.
Months 12–14	Evaluation, safety and observability.	Publish repeatable quality and performance results.
Months 14–16	Deployment and production operations.	Deploy a secure, monitored application.
Months 16–18	Portfolio, interview preparation and specialization.	Present two or three credible project case studies.

Timeline disclaimer: Completing this plan does not guarantee employment. Hiring depends on role requirements, prior experience, communication, interview performance, location and market conditions. Use the checkpoints to judge readiness rather than the calendar alone.

13. After the roadmap: choose a specialization

Once you can build and ship reliably, go deep in one direction. Pick the track that matches the job descriptions you want.

Track	Go deeper in	Strong next project
LLM application engineer	Context design, RAG quality, agents and product UX.	A multi-tenant assistant with per-user permissions and evaluation.
Machine learning engineer	Training pipelines, experiment tracking and model serving.	A retrainable model service with drift monitoring.
Inference and platform engineer	Batching, quantization, GPU use, queues and autoscaling.	A load-tested model-serving deployment with a cost report.
AI quality and safety engineer	Evaluation design, red teaming and guardrails.	An open evaluation harness for a RAG or agent system.
Data and retrieval engineer	Ingestion pipelines, indexing, search relevance and data quality.	A search service with freshness handling and relevance metrics.

Interview questions to practice

Technical

How would you design a RAG system for 100,000 documents with per-user access?
How do you detect and reduce hallucinations?
When would you fine-tune instead of improving retrieval?
How do you defend an agent against prompt injection?
How would you cut cost and latency without losing quality?

Project and behavioral

What failed in your project, and what did you change?
How did you measure that version two was better?
Which trade-off did you make between quality, cost and speed?
Describe a bug you debugged in a model-backed feature.
What would you improve with another month?

14. Official documentation and learning resources

Prefer primary documentation for tools and concepts. Check the current version and pricing before choosing a dependency for a project.

Python Tutorial

Language fundamentals and standard-library concepts.

FastAPI Documentation

Build APIs, validate requests and understand dependency injection.

PyTorch Tutorials

Tensor operations, model building and training workflows.

scikit-learn User Guide

Classical machine learning, preprocessing and evaluation.

Hugging Face Learn

Practical courses on transformers, datasets and model tooling.

OpenAI API Documentation

API concepts, structured outputs and model integration.

Model Context Protocol

Understand the protocol for connecting AI applications to tools and context.

Docker Get Started

Container fundamentals for reproducible development and deployment.

OpenTelemetry Documentation

Vendor-neutral observability concepts, traces, metrics and logs.

OWASP Top 10 for LLM Applications

Security risks and defensive practices for LLM-powered systems.

Pydantic Documentation

Data validation and schemas for structured model output.

pgvector

Vector similarity search inside PostgreSQL.

Frequently asked questions

Can I become an AI engineer without a computer science degree?

It is possible, but you still need to demonstrate the underlying skills. Build strong programming fundamentals, learn the relevant ML and AI concepts, and publish projects that show engineering quality. Some employers may require or prefer a degree, so review the requirements of your target roles.

Do I need to learn advanced mathematics?

For application-focused roles, practical knowledge of linear algebra, probability, statistics and optimization is often enough to begin. Model development and research roles generally require deeper mathematical understanding.

Should I learn RAG or fine-tuning first?

Start with prompting, structured outputs and RAG when the application needs external or private knowledge. Fine-tuning is useful for some adaptation problems, but it should be driven by evaluation evidence rather than used by default.

Do I need a powerful GPU?

No. You can learn programming, APIs, evaluation and many LLM application patterns using hosted model APIs or small local models. A GPU becomes more important for larger-scale local inference, training and performance experimentation.

How many projects should I put in my portfolio?

Two or three complete, well-documented projects are usually more persuasive than many unfinished demos. Make your contribution, evaluation method, limitations and deployment approach easy to understand.

Which programming language should I learn first?

Python is a practical first choice because of its ecosystem for data science, machine learning and AI application development. Learn SQL and enough JavaScript or TypeScript to integrate with the front end when your projects require it.

Is learning a specific AI framework mandatory?

No single framework is mandatory across the industry. Learn the underlying patterns—model calls, retrieval, tool execution, evaluation and observability—then select a framework when it improves maintainability or development speed.

Is an AI engineer the same as a prompt engineer?

No. Prompt design is one skill inside AI engineering. AI engineers also build backends, retrieval systems, tool integrations, evaluation suites, security controls and deployment pipelines.

How do I get my first AI engineering role?

Publish two or three documented projects with evaluation results, practice coding and system-design interviews, and apply to roles whose responsibilities match your skills. Internships, open-source contributions and internal transfers can also open doors.

How do I keep up without chasing every new release?

Follow a few primary sources such as official documentation and release notes, and re-run your evaluation set when you try a new model or tool. Adopt a change only when your own measurements show it helps.

How much does it cost to learn AI engineering?

Most documentation and many courses are free. Hosted model APIs charge by usage, so set spending limits, use small test sets and cache results. A powerful GPU is not required to start.

Will AI coding assistants replace AI engineers?

Assistants change how code is written, but someone still has to define the problem, review the output, evaluate quality, secure the system and own it in production. Use assistants to move faster, and make sure you can explain and debug every line you ship.

Conclusion: your next steps

Becoming an AI engineer is less about knowing every model and more about repeatedly shipping small systems that work, measuring them and improving them. The roadmap gives you the order. Your projects give you the proof.

This week

Pick your starting point, set up Python, Git and a GitHub repository, and complete the Week 1 deliverable.

This month

Finish the first 30 days and choose the capstone problem you will keep improving for the next eight months.

This quarter

Pass the checkpoints for Phases 1–3, publish your first documented project and write down what you learned.

Key takeaways

AI engineering is software engineering plus models, data, evaluation and security.
Learn in layers, and use checkpoints instead of the calendar to judge progress.
Start with prompting, structured output and RAG before fine-tuning or autonomous agents.
Validate model output, limit tool permissions and treat all retrieved text as untrusted.
Measure everything: quality, latency, cost and failure modes.
Ship two or three documented projects instead of many unfinished demos.

Learn the foundations. Build a real product. Evaluate it with evidence. Secure it. Deploy it. Improve it. That is the practical path from learning AI concepts to demonstrating AI engineering ability.

About this roadmap: This is a learning guide, not a promise of employment or a fixed industry standard. AI tooling and role expectations evolve quickly. Re-check official documentation, job descriptions and security guidance before making technology decisions.

How to Build a PDF Merger in Python with Streamlit and pypdf

2026-10-27T22:35:03.254+05:30

Build a PDF Merger in Python with Streamlit & pypdf | RustCode

How to Build a PDF Merger in Python with Streamlit and pypdf

This guide explains a complete PDF merger web app, one section at a time. Users upload PDFs, reorder them, choose pages from each file, and download a single merged document.

What the app does

The app is a single Python file built with Streamlit for the interface and pypdf for reading and writing PDFs. Everything runs in memory, so no files are saved to disk.

Upload any number of PDF files at once.
Move files up or down to set their order in the final document.
Pick pages for each file, such as 1-3 or 5,1,3.
Validate every input before anything is merged.
Download the result under a safe filename.

The script flows from top to bottom: upload → read → order → select pages → merge → download. Streamlit re-runs the whole script after every interaction, which explains several design choices below.

Install and run

You need Python 3.9 or newer. Install the two dependencies and start the app:

pip install streamlit pypdf
streamlit run pdf_merge.py

Streamlit opens the app in your browser, usually at http://localhost:8501.

Imports and page setup

import io
import re

import streamlit as st
from pypdf import PdfReader, PdfWriter

st.set_page_config(page_title="PDF Merger", page_icon="📄", layout="wide")
st.title("📄 PDF Merger")

io provides BytesIO, which lets pypdf read uploaded bytes as if they were a file.
re is used later to clean the output filename.
PdfReader reads pages, and PdfWriter builds the merged file.
st.set_page_config must be the first Streamlit call. It sets the browser tab title, icon, and wide layout.

The script then shows a short description and a caption with page selection examples, so users know the syntax before they type.

Parsing page selections

The function parse_page_selection turns text like 2-4,7 into a list of zero-based page indexes. Zero-based indexing matches how pypdf stores pages, while users still type page numbers starting at 1.

Handling "all" and basic checks

def parse_page_selection(selection: str, total_pages: int) -> list[int]:
    selection = (selection or "all").strip().lower()

    if selection in {"all", "*"}:
        return list(range(total_pages))
    if not selection:
        raise ValueError("Enter `all` or one or more page numbers/ranges.")

    pages: list[int] = []
    seen: set[int] = set()

Empty input defaults to all. The pages list keeps the requested order, and the seen set prevents duplicates.

Splitting the input into parts

    for part in selection.split(","):
        part = part.strip()
        if not part:
            raise ValueError("There is an empty item in the page selection.")

        if "-" in part:
            bits = part.split("-")
            if len(bits) != 2 or not all(bit.strip().isdigit() for bit in bits):
                raise ValueError(f"Invalid page range: `{part}`.")
            start, end = (int(bit.strip()) for bit in bits)
            if start < 1 or end < 1:
                raise ValueError("Page numbers must be 1 or greater.")
            if start > end:
                raise ValueError(f"Range `{part}` must be in ascending order.")
            requested = range(start, end + 1)
        else:
            if not part.isdigit():
                raise ValueError(f"Invalid page number: `{part}`.")
            requested = [int(part)]

Each comma-separated part is either a range (2-4) or a single page (7). Anything else, such as letters or 1-2-3, raises a clear error.

Range check and duplicates

        for page in requested:
            if page < 1 or page > total_pages:
                raise ValueError(
                    f"Page {page} is out of range. This PDF has {total_pages} page(s)."
                )
            if page not in seen:
                pages.append(page - 1)
                seen.add(page)

    if not pages:
        raise ValueError("No pages were selected.")
    return pages

Every page is checked against the PDF length. Valid pages are converted to zero-based indexes and added once.

Input	Meaning	Result (zero-based)
`all`	Every page	All pages in order
`1-3`	Pages 1 to 3	`[0, 1, 2]`
`1,3,5`	Individual pages	`[0, 2, 4]`
`5,1,3`	Custom order	`[4, 0, 2]`
`2-4,7`	Range plus one page	`[1, 2, 3, 6]`

Cleaning the output filename

def safe_filename(name: str) -> str:
    name = (name or "").strip()
    if not name:
        name = "merged.pdf"
    if not name.lower().endswith(".pdf"):
        name += ".pdf"
    cleaned = re.sub(r"[^A-Za-z0-9._-]+", "_", name)
    cleaned = cleaned.strip("._")
    if not cleaned:
        cleaned = "merged.pdf"
    if not cleaned.lower().endswith(".pdf"):
        cleaned += ".pdf"
    return cleaned

This function guarantees a safe download name:

It falls back to merged.pdf when the field is empty.
It adds the .pdf extension if missing.
It replaces unusual characters with underscores, so my report (v2) becomes my_report_v2_.pdf.
It trims leading and trailing dots and underscores, then checks the extension once more.

Uploading and reading PDFs

Session state and the uploader

if "pdf_order" not in st.session_state:
    st.session_state.pdf_order = []

uploads = st.file_uploader(
    "Choose PDF files",
    type=["pdf"],
    accept_multiple_files=True,
)

if not uploads:
    st.info("Upload one or more PDFs to get started.")
    st.stop()

Because Streamlit re-runs the script on every click, the file order must live in st.session_state. Otherwise it would reset each time. st.stop() ends the run early until at least one file is uploaded.

Reading each file safely

current_items = []
for i, upload in enumerate(uploads):
    key = f"{upload.name}::{upload.size}::{i}"
    raw = upload.getvalue()

    try:
        reader = PdfReader(io.BytesIO(raw), strict=False)
        if reader.is_encrypted:
            try:
                result = reader.decrypt("")
                if not result:
                    st.error(f"`{upload.name}` is password-protected. ...")
                    continue
            except Exception:
                st.error(f"`{upload.name}` is password-protected. ...")
                continue

        page_count = len(reader.pages)
        if page_count == 0:
            st.warning(f"`{upload.name}` contains no pages and will be skipped.")
            continue

        current_items.append({
            "key": key, "name": upload.name, "size": upload.size,
            "bytes": raw, "pages": page_count,
        })
    except Exception as exc:
        st.error(f"Could not read `{upload.name}`: {exc}")

The key combines name, size, and position, so two files with the same name stay distinct.
strict=False lets pypdf open slightly damaged PDFs instead of failing.
Encrypted files are tested with an empty password. If that fails, the user sees a clear message and the file is skipped.
Empty PDFs are skipped with a warning, and unreadable files show the error without crashing the app.

Valid files are stored as dictionaries holding the raw bytes and page count for later steps.

Reordering files

item_by_key = {item["key"]: item for item in current_items}
current_keys = [item["key"] for item in current_items]

old_order = [key for key in st.session_state.pdf_order if key in item_by_key]
new_keys = [key for key in current_keys if key not in old_order]
st.session_state.pdf_order = old_order + new_keys

This block syncs saved order with the current uploads. Files the user removed disappear, files that remain keep their position, and new uploads are added at the end.

for position, key in enumerate(st.session_state.pdf_order):
    item = item_by_key[key]
    left, middle, up_col, down_col = st.columns([5, 2, 1, 1])
    left.markdown(f"**{position + 1}. {item['name']}**")
    middle.caption(f"{item['pages']} page(s) · {item['size'] / 1024:.1f} KB")

    if up_col.button("↑", key=f"up_{key}", disabled=(position == 0)):
        order = st.session_state.pdf_order.copy()
        order[position - 1], order[position] = order[position], order[position - 1]
        st.session_state.pdf_order = order
        st.rerun()

Each file gets a row with its name, page count, size, and two buttons. A button swaps the file with its neighbor, saves the new order, and calls st.rerun() so the list refreshes immediately. The up button is disabled for the first file, and the down button works the same way, disabled for the last file.

Choosing pages per file

page_selections: dict[str, str] = {}

for position, key in enumerate(st.session_state.pdf_order):
    item = item_by_key[key]
    with st.container(border=True):
        st.markdown(f"**{position + 1}. {item['name']}**")
        st.caption(f"Total pages: {item['pages']}")
        page_selections[key] = st.text_input(
            "Pages to include",
            value="all",
            key=f"pages_{key}",
            placeholder="all, 1-3, 1,3,5, or 5,1,3",
        )

Every file gets a bordered card with a text box that defaults to all. The text is stored by file key, so it stays attached to the right PDF even after reordering. Nothing is validated here. That happens when the user clicks Merge.

Merging and downloading

The output filename and Merge button

output_name_input = st.text_input("Output filename", value="merged.pdf")
output_name = safe_filename(output_name_input)

if st.button("Merge PDFs", type="primary", use_container_width=True):

Validate first

    validated_selections = {}
    errors = []

    for key in st.session_state.pdf_order:
        item = item_by_key[key]
        try:
            validated_selections[key] = parse_page_selection(
                page_selections.get(key, "all"), item["pages"]
            )
        except ValueError as exc:
            errors.append(f"{item['name']}: {exc}")

    if errors:
        st.error("Please correct these page-selection errors:\n\n" + "\n\n".join(errors))

All selections are checked before any output is created. If several files have mistakes, the user sees every error at once instead of fixing them one by one.

Build the merged PDF

    else:
        writer = PdfWriter()
        total_added = 0

        try:
            for key in st.session_state.pdf_order:
                item = item_by_key[key]
                reader = PdfReader(io.BytesIO(item["bytes"]), strict=False)

                if reader.is_encrypted:
                    result = reader.decrypt("")
                    if not result:
                        raise ValueError(f"{item['name']} is encrypted ...")

                for page_index in validated_selections[key]:
                    writer.add_page(reader.pages[page_index])
                    total_added += 1

            output = io.BytesIO()
            writer.write(output)
            output_bytes = output.getvalue()

The code loops through files in the chosen order and adds the selected pages one by one. The result is written to an in-memory buffer rather than a file on disk.

Show the result

            st.success(
                f"Successfully merged {total_added} page(s) from "
                f"{len(st.session_state.pdf_order)} PDF file(s)."
            )
            st.download_button(
                label=f"Download {output_name}",
                data=output_bytes,
                file_name=output_name,
                mime="application/pdf",
                use_container_width=True,
            )
        except Exception as exc:
            st.error(f"Could not create the merged PDF: {exc}")

On success, the user sees a summary and a download button. Any unexpected failure is caught and shown as a readable error.

Ideas for improvement

Password support: add a password field so encrypted PDFs can be unlocked.
Page previews: render thumbnails so users can pick pages visually.
Reverse and rotate options: pypdf can rotate pages with page.rotate(90).
Bookmarks: add an outline entry for each source file with writer.add_outline_item().
Size limits: warn when total upload size is large, since all data stays in memory.

Conclusion

This app shows how a small amount of Python can deliver a practical tool. Streamlit handles the interface, session state keeps the order stable, and pypdf does the merging. Validating first, handling errors clearly, and sanitizing filenames keep it reliable. Copy the sections above into one file, run it, and adapt it to your own workflow.

How To Generate a Random Color in JavaScript

2026-05-09T10:36:15.680+05:30

How To Generate a Random Color in JavaScript

Learn clean, efficient, and professional techniques to generate random colors for modern web applications

Understanding Random Color Generation

JavaScript provides powerful built-in tools like Math.random() to create dynamic and engaging visual experiences. Random colors are widely used in data visualizations, interactive backgrounds, games, generative art, and UI enhancements.

The most common color formats in CSS are:

Hexadecimal (#RRGGBB) — Most popular and compact format
RGB/RGBA — Uses decimal values from 0 to 255
HSL/HSLA — Hue, Saturation, Lightness (more human-friendly)

Fastest Hexadecimal Method (Recommended)

const randomHex = Math.floor(Math.random() * 16777215).toString(16).padStart(6, '0');
document.body.style.backgroundColor = '#' + randomHex;

Output Example: #7b4cff

Code Explanation:

Math.random() * 16777215: Generates a random number between 0 and 16,777,215 (which is FFFFFF in hex).
Math.floor(): Rounds down to the nearest whole number.
.toString(16): Converts the number into hexadecimal format.
.padStart(6, '0'): Ensures the result is always 6 characters long by adding zeros at the beginning if needed.
Finally, we add '#' prefix and apply it to the background.

Best for: Performance-critical applications and simple use cases.

Recursive Character Builder

const generateHex = (code = '') => {
  const chars = '0123456789abcdef';
  code += chars[Math.floor(Math.random() * 16)];
  return code.length === 6 ? code : generateHex(code);
};

const color = generateHex();

Output Example: #a1f34c

Code Explanation:

Uses recursion to build the 6-character hex code one digit at a time.
chars[Math.floor(Math.random() * 16)]: Picks a random character from 0-9 and a-f.
Keeps calling itself until the string length reaches 6.
More readable and easier to extend (e.g., for 8-character colors with alpha).

Best for: Learning recursion and when you need more control over color generation.

Functional Recursive Approach

const randomColor = (function generate(m, c, l) {
  return l ? generate(m, c, l - 1) + c[Math.floor(m.random() * c.length)] : '#';
})(Math, '0123456789ABCDEF', 6);

Output Example: #FF6B6B

Code Explanation:

Advanced IIFE (Immediately Invoked Function Expression) with recursion.
Builds the color string from left to right using recursion.
When l (length) becomes 0, it returns '#' as base.
Each recursive call adds one random character.

Best for: Functional programming enthusiasts and code golf / elegant solutions.

RGB Method

const r = Math.floor(Math.random() * 256);
const g = Math.floor(Math.random() * 256);
const b = Math.floor(Math.random() * 256);
document.body.style.backgroundColor = `rgb(${r}, ${g}, ${b})`;

Output Example: rgb(124, 45, 210)

Code Explanation:

Generates random values for Red, Green, and Blue channels (0–255).
Uses template literals to create rgb(r, g, b) format.
More readable than hex for some developers.
Easier to modify for RGBA (adding transparency).

Best for: When you need to work with individual color channels.

The Evolution of Artificial Intelligence: Timeline of Innovation from 1950s to Today

2026-04-25T20:56:46.698+05:30

History of Artificial Intelligence - Complete Timeline | Rustcode

The Rise of Artificial Intelligence

A Complete Timeline from Symbolic Reasoning to Generative AI & Large Language Models

Visual timeline: Major AI eras — 1950s-60s (Symbolic AI), 1970s-80s (Expert Systems), 1990s (Machine Learning), 2000s (Deep Learning), 2010s–present (Generative AI & LLMs)

Introduction

Artificial Intelligence (AI) has transformed from a philosophical concept into the defining technological force of the 21st century. Beginning with the Dartmouth Workshop in 1956, AI has experienced waves of optimism, "AI winters," and unprecedented breakthroughs. This timeline traces the pivotal moments, researchers, algorithms, and systems that shaped modern AI — from early logical reasoning and expert systems to deep learning, transformers, and generative models that redefine human-computer interaction.

Year	Milestone / Innovation	Creator(s) / Institution	Key Contribution	Impact
The Birth of AI (1940s–1950s)
1950	Turing Test	Alan Turing	Proposed criteria for machine intelligence: "Can machines think?"	Foundational Established philosophical and practical benchmark for AI.
1956	Dartmouth Workshop	John McCarthy, Marvin Minsky, Claude Shannon, Nathaniel Rochester	Coined term "Artificial Intelligence"; birth of AI as a field.	Foundational Unified early AI research, leading to symbolic AI.
1958	LISP	John McCarthy	Programming language designed for AI research and symbolic computation.	Foundational Dominant AI language for decades; enabled early reasoning systems.
Early Enthusiasm & Expert Systems (1960s–1970s)
1966	ELIZA	Joseph Weizenbaum (MIT)	First natural language processing chatbot simulating Rogerian therapist.	Foundational Pioneered human-computer dialogue and pattern matching.
1972	MYCIN	Edward Shortliffe (Stanford)	Early rule-based expert system for bacterial infection diagnosis.	Specialized Demonstrated AI in medicine and reasoning under uncertainty.
1979	Stanford Cart	Hans Moravec (Stanford)	First autonomous vehicle using 3D vision and obstacle avoidance.	Foundational Precursor to modern self-driving systems.
Knowledge-Based Systems & AI Winters (1980s)
1986	Backpropagation Renaissance	Rumelhart, Hinton, Williams	Popularized backpropagation for training multi-layer neural networks.	Foundational Revived neural network research, enabling deep learning.
1988	Hidden Markov Models (HMMs) for Speech	AT&T Bell Labs (Rabiner, Juang)	Statistical model for temporal pattern recognition in speech.	Specialized Became backbone of speech recognition systems.
Machine Learning & Statistical Revolution (1990s)
1997	IBM Deep Blue vs. Kasparov	IBM (Hsu, Campbell)	First computer to defeat a reigning world chess champion under tournament conditions.	Foundational Mainstream validation of search + evaluation functions.
1998	LeNet-5	Yann LeCun, Yoshua Bengio et al.	Convolutional neural network (CNN) for handwritten digit classification.	Foundational Blueprint for modern deep computer vision.
Deep Learning Breakthroughs (2000s–2010)
2006	Deep Belief Networks	Geoffrey Hinton, Simon Osindero, Yee-Whye Teh	Layer-wise pretraining unlocking deep neural network potential.	Foundational Sparked modern deep learning revolution.
2012	AlexNet	Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton	CNN wins ImageNet with dramatic error reduction using GPUs.	Modern Catalyzed computer vision and deep learning adoption.
Generative AI & Large Language Models (2014–Present)
2014	Generative Adversarial Networks (GANs)	Ian Goodfellow et al.	Two-network adversarial training generates realistic synthetic data.	Modern Revolutionized image generation, deepfakes, art.
2017	Transformer Architecture	Vaswani et al. (Google Brain)	“Attention Is All You Need” — self-attention mechanism for sequence tasks.	Foundational Foundation of GPT, BERT, and all modern LLMs.
2018	BERT & Pre-training Revolution	Google (Devlin et al.)	Bidirectional encoder representations from transformers for NLP.	Modern Set state-of-the-art benchmarks across language understanding.
2020	GPT-3	OpenAI (Brown et al.)	175 billion parameter LLM demonstrating few-shot in-context learning.	Modern Mainstream generative AI: creative writing, coding, reasoning.
2022	ChatGPT & Instruction Tuning	OpenAI	Conversational agent based on GPT-3.5/4 with RLHF alignment.	Modern Over 100M users, triggered global generative AI race.
2023	GPT-4 & Multimodal AI	OpenAI	Large multimodal model accepts image/text inputs, human-level exam performance.	Modern Advanced reasoning, reduced hallucinations, integration into tools.
2024–2025	Agentic AI & Real-time Systems	OpenAI, Google DeepMind, Anthropic, xAI	LLM-based agents with tool use, memory, multi-step reasoning; AI assistants integrated into daily workflows.	Modern Shift from chatbots to autonomous task-completion agents.

Key Milestones in AI Evolution

1956 – Dartmouth Workshop: Officially launches AI as a discipline, nurturing early symbolic AI and logic-based reasoning.
1997 – Deep Blue vs. Kasparov: AI defeats world chess champion, showcasing brute-force search and heuristics on supercomputers.
2012 – AlexNet: Deep learning breakthrough proves GPU-accelerated neural networks dominate computer vision, triggering corporate investment.
2017 – Transformer Architecture: “Attention Is All You Need” replaces RNNs, enabling massive parallelization and scaling of language models.
2020 – GPT-3: Demonstrates emergent abilities in few-shot learning, code generation, and creative tasks, redefining NLP limits.
2022 – ChatGPT: Conversational AI brings LLMs to mainstream consumers; widespread adoption across industries.
2023 – GPT-4: Multimodal capabilities and advanced reasoning marks the arrival of human-competitive AI in standardized tests.

Modern AI Trends & Paradigms

Contemporary AI research is defined by generative capabilities, agentic workflows, multimodal integration, and efforts toward safe, aligned superhuman intelligence. The following trends shape today's frontier:

Generative AI & Foundation Models

Models like GPT-4, Stable Diffusion, and Sora generate coherent text, images, video, and code. Pre-training on internet-scale data enables zero-shot generalization across domains.

AI Agents & Tool Use

LLM-based agents plan, execute actions, browse web, use APIs, and collaborate. Frameworks like AutoGPT, LangChain enable autonomous research, software development, and operations.

Multimodal Intelligence

Unified models (GPT-4V, Gemini, LLaVA) process images, audio, video, and text jointly, enabling real-world understanding — from self-driving to medical diagnosis.

Alignment & AI Safety

Reinforcement Learning from Human Feedback (RLHF), Constitutional AI, and scalable oversight aim to ensure systems act ethically, truthfully, and robustly without harmful side effects.

Edge & Tiny AI

Efficient models (MobileBERT, TinyML, On-Device LLMs) bring inference to smartphones, IoT, and embedded systems, reducing latency and preserving privacy.

Neuro-Symbolic & Causal AI

Hybrid systems combining deep learning with symbolic reasoning offer interpretability, data efficiency, and causal inference — closing gaps in pure neural approaches.

Influence and Legacy of AI

Artificial intelligence has reshaped every facet of science and industry. The journey from logic theorems and chess engines to generative chatbots and autonomous agents reveals not only accelerating progress but also enduring challenges. Landmark innovations — symbolic LISP, expert systems, deep learning, transformers — each contributed layers of capability. Today’s intelligent systems inherit principles from decades of research while driving next-generation breakthroughs in biology, robotics, creativity, and decision-making.

Foundational Paradigms

Symbolic AI (expert systems, logic programming) laid the foundation for reasoning and knowledge representation. Neural networks and backpropagation introduced learning from data — both are now fused in hybrid models.

Cross-Disciplinary Impact

AI methods now drive computational biology (AlphaFold), materials science, climate modeling, personalized medicine, and autonomous systems — blending research domains that were once isolated.

Modern Innovation Engines

The transformer architecture became the backbone of LLMs, vision transformers (ViT), and multimodal models. AI scaling laws reveal path toward broader artificial general intelligence capabilities.

Future Horizons

Emerging trends like self-improving agents, continual learning, and world models move AI toward adaptability, common sense, and trustworthiness. Ethical frameworks and global governance will define AI's long-term coexistence with humanity.

Sources & Further Reading:

Constructing Documents in Mongoose

2025-09-29T00:27:00.002+05:30

Constructing Documents in Mongoose | Rustcode

Constructing Documents in Mongoose

In Mongoose, documents are instances of models and represent individual records in your MongoDB collections. Constructing documents involves creating instances of a model, initializing them with data, and optionally saving them to the database.

Creating a Document Instance

To construct a new document, use the new keyword with your compiled model:

const user = new User({
  name: { first: 'Jane', last: 'Doe' },
  email: 'jane.doe@example.com',
  age: 28
});

Setting and Modifying Properties

You can set properties directly via the constructor or modify the instance properties later:

// Initial properties at construction time
const post = new Post({ title: 'Hello World', content: 'My first post' });

// Modifying properties after construction
post.title = 'Updated Title';
post.content = 'Updated content';

Saving Documents to MongoDB

To persist your constructed document to the database, call the asynchronous save() method:

user.save()
  .then(doc => console.log('Document saved:', doc))
  .catch(err => console.error('Save error:', err));

Using Document Methods and Virtuals

Documents can use instance methods and virtual properties defined on the schema. This enables custom behavior and computed values.

user.fullName();   // Example instance method
console.log(user.fullName); // Example virtual property

Summary Table

Operation	Description	Code Example
Create	Initialize a new document instance	`new Model(data)`
Modify	Update document properties	`doc.property = value`
Save	Persist document to MongoDB	`doc.save()`

Best Practices

Validate data before saving documents.
Use schema methods and virtuals to encapsulate logic.
Handle errors during save operations gracefully.
Create documents using model constructors consistently.

Conclusion

Constructing documents through Mongoose models provides a powerful and flexible way to manage MongoDB records. With support for instance methods and schema validations, your documents stay consistent and your data layer clean.

Compiling Your First Model in Mongoose

2025-09-28T00:25:00.001+05:30

Compiling Your First Model in Mongoose | Rustcode

Compiling Your First Model in Mongoose

In Mongoose, a model is a compiled representation of a schema. Models act as constructors for documents and provide an interface to interact with the corresponding MongoDB collections.

Defining a Schema

Before compiling a model, you need to define a schema that maps to the structure of documents in your MongoDB collection. The schema defines fields, data types, validations, and other constraints.

const mongoose = require('mongoose');

const userSchema = new mongoose.Schema({
  name: {
    first: String,
    last: String
  },
  email: {
    type: String,
    required: true,
    unique: true
  },
  age: Number,
  createdAt: {
    type: Date,
    default: Date.now
  }
});

Compiling the Model

Use the mongoose.model() function to compile your schema into a model. The first argument is the string name of the model (and implicitly the MongoDB collection name, pluralized), and the second is the schema.

const User = mongoose.model('User', userSchema);

The model name 'User' corresponds to the 'users' collection in MongoDB by default.

Using the Model

Once compiled, you can use the model to create, read, update, and delete documents.

// Creating a new user document
const newUser = new User({
  name: { first: 'John', last: 'Doe' },
  email: 'john.doe@example.com',
  age: 30
});

newUser.save()
  .then(doc => console.log('User saved:', doc))
  .catch(err => console.error('Error saving user:', err));

Summary Table

Step	Description	Code Example
Define Schema	Map document structure and validations	`new mongoose.Schema({ ... })`
Compile Model	Create model constructor from schema	`mongoose.model('Name', schema)`
Use Model	Create and query MongoDB documents	`new Model()` and `save()`

Best Practices

Define clear and consistent schemas.
Use schema validations to keep data integrity.
Compile models once per schema and reuse them.
Use meaningful model names aligned with collection content.

Conclusion

Compiling your first model in Mongoose is the foundational step to interacting with MongoDB collections in a structured and efficient way. Clear schemas and well-defined models enable reliable data management in your applications.

TLS/SSL Connections in Mongoose

2025-09-27T00:01:00.001+05:30

TLS/SSL Connections in Mongoose | Rustcode

Securing Mongoose Connections with TLS/SSL

TLS (Transport Layer Security) and its predecessor SSL (Secure Sockets Layer) provide encryption and secure communication between your Mongoose application and MongoDB servers. Configuring TLS/SSL connections prevents eavesdropping, man-in-the-middle attacks, and data tampering.

Enabling TLS/SSL in Mongoose

To enable TLS/SSL, you can add options to your connection configuration. When using MongoDB Atlas or other providers, TLS is often enabled by default.

mongoose.connect('mongodb://host:27017/mydb', {
  useNewUrlParser: true,
  useUnifiedTopology: true,
  tls: true,
  tlsCAFile: '/path/to/ca.pem',
  tlsCertificateKeyFile: '/path/to/client.pem',
  tlsCertificateKeyFilePassword: 'yourPassword' // if your certificate is password protected
});

Paths in options like tlsCAFile and tlsCertificateKeyFile refer to certificate files used to validate the server and provide client authentication.

Important TLS/SSL Options

tls: Enables TLS/SSL connection (boolean).
tlsCAFile: Path to the Certificate Authority file to validate server certificates.
tlsCertificateKeyFile: Client certificate and key for authentication.
tlsCertificateKeyFilePassword: Password for the client certificate file.
tlsAllowInvalidCertificates: Allow connections with invalid certificates (not recommended for production).
tlsAllowInvalidHostnames: Allow connections if hostnames do not match certificate.

Example Connecting to MongoDB Atlas

MongoDB Atlas provides a connection string with TLS enabled by default.

mongoose.connect('mongodb+srv://username:password@cluster0.mongodb.net/mydb?retryWrites=true&w=majority', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

Security Best Practices

Always enable TLS/SSL in production environments.
Use trusted CA certificates to avoid man-in-the-middle attacks.
Protect private keys with strong passwords and secure file permissions.
Avoid disabling validation options like tlsAllowInvalidCertificates except in trusted development environments.

Summary Table

Option	Purpose	Notes
`tls`	Enable TLS/SSL	Set to true for secure connections
`tlsCAFile`	CA certificate to validate server	Use trusted CA certificates
`tlsCertificateKeyFile`	Client certificate and key	For client authentication if required
`tlsAllowInvalidCertificates`	Allow invalid certificates	Disable in production

Conclusion

Configuring TLS/SSL connections in Mongoose is critical for securing data in transit. Proper certificate management and enabling encryption ensure your MongoDB communications remain private and trusted.

Multi Tenant Connections in Mongoose

2025-09-26T23:57:00.004+05:30

Multi Tenant Connections in Mongoose | Rustcode

Implementing Multi Tenant Connections in Mongoose

Multi-tenant applications serve multiple customers (tenants) using a single instance of the application and often require strict data isolation. Mongoose supports multi-tenant architectures primarily by managing separate connections per tenant, enabling isolated databases or clusters.

Why Use Separate Connections per Tenant?

Using separate connections for each tenant provides:

Data isolation and security by separating databases.
Improved scalability, allowing different tenants to be served from different clusters or database instances.
Granular control over connection settings per tenant.

How to Manage Multi Tenant Connections

Use mongoose.createConnection() to create and store a connection instance per tenant. For example:

const connections = {};

function getTenantConnection(tenantId) {
  if (!connections[tenantId]) {
    connections[tenantId] = mongoose.createConnection(`mongodb://localhost:27017/${tenantId}`, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
    });
  }
  return connections[tenantId];
}

// Usage
const tenantDb = getTenantConnection('tenantA');
const User = tenantDb.model('User', userSchema);

Cache connections per tenant to avoid unnecessary reconnections and improve performance.

Considerations and Challenges

Managing connection lifecycles to avoid resource exhaustion.
Scaling connection pools for many tenants.
Handling tenant-specific configurations or middlewares.

Summary Table

Aspect	Description	Notes
Separate connections	One connection instance per tenant	Ensures isolation and scalability
Connection caching	Reuse existing connections	Improves performance
Resource management	Monitor connection limits	Essential for large tenant counts

Best Practices

Cache connections and reuse them to reduce overhead.
Implement proper error handling and reconnection strategies.
Monitor resource usage and scale your MongoDB instances accordingly.
Consider using sharded clusters or dedicated clusters for heavy tenants.

Conclusion

Mongoose facilitates multi-tenant application development through isolated connections per tenant. Managing these connections effectively is key to building scalable, secure, and maintainable multi-tenant systems.

Connection Pools in Mongoose

2025-09-25T23:54:00.001+05:30

Connection Pools in Mongoose | Rustcode

Understanding Connection Pools in Mongoose

Connection pooling is a technique used to maintain multiple open connections to the MongoDB server, allowing your Mongoose application to reuse existing connections efficiently and improve performance.

What is a Connection Pool?

Rather than opening and closing a new connection for every database operation, a connection pool keeps a set of connections open. This reduces connection overhead, latency, and resource consumption by reusing these connections for multiple requests.

Configuring Connection Pool Size

You can configure the maximum number of connections in the pool using the maxPoolSize option when connecting:

mongoose.connect('mongodb://localhost:27017/mydb', {
  useNewUrlParser: true,
  useUnifiedTopology: true,
  maxPoolSize: 10  // Default is 5 or 100 depending on the driver version
});

Adjusting maxPoolSize helps balance performance and resource usage based on your application's workload.

Other Pool-Related Options

minPoolSize: Minimum number of connections to keep open in the pool.
maxIdleTimeMS: Time a connection can remain idle in the pool before being closed.
waitQueueTimeoutMS: Maximum time an operation waits for an available connection from the pool.

Benefits of Using Connection Pools

Reduced latency from connection reuse.
Better resource management on the MongoDB server and client.
Improved scalability of your application.

Summary Table

Option	Description	Default
`maxPoolSize`	Maximum number of connections in the pool	5 (or 100 in recent driver versions)
`minPoolSize`	Minimum pool size maintained	0
`maxIdleTimeMS`	Idle time before closing a connection	0 (no timeout)
`waitQueueTimeoutMS`	Max time an operation waits for a connection	0 (no timeout)

Best Practices

Set maxPoolSize based on expected application load and server capacity.
Monitor connection pool usage with MongoDB monitoring tools.
Use the useUnifiedTopology option to leverage the latest server discovery and monitoring features.

Conclusion

Connection pooling is essential for building efficient and scalable Mongoose applications. Proper configuration ensures your app can handle increased load while efficiently using MongoDB resources.

Multiple Connections in Mongoose

2025-09-24T23:52:00.002+05:30

Multiple Connections in Mongoose | Rustcode

Managing Multiple Connections in Mongoose

In some applications, you may need to connect to multiple MongoDB databases or multiple MongoDB clusters simultaneously. Mongoose supports this via its createConnection() method that allows you to establish and manage multiple distinct connections.

Using mongoose.createConnection()

Unlike mongoose.connect(), which sets up a default connection, createConnection() returns a separate connection instance. You can use this instance to define models and perform operations independently.

const mongoose = require('mongoose');

const conn1 = mongoose.createConnection('mongodb://localhost:27017/dbOne', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

const conn2 = mongoose.createConnection('mongodb://localhost:27017/dbTwo', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

// Define models on each connection separately
const User = conn1.model('User', userSchema);
const Product = conn2.model('Product', productSchema);

Using multiple connections allows finer control when working with different databases or clusters.

When to Use Multiple Connections

Connecting to multiple databases within the same app.
Separating concerns by database or service.
Working with different MongoDB clusters or cloud providers.

Connection Lifecycle Management

Each connection instance acts independently; you must manage events like open, error, and closing for each connection.

conn1.on('open', () => {
  console.log('Connection 1 is open');
});

conn2.on('error', (err) => {
  console.error('Connection 2 error:', err);
});

Summary Table

Feature	Description	Notes
`mongoose.connect()`	Creates a default, global connection	Used when connecting to a single database
`mongoose.createConnection()`	Creates distinct connection instances	Use for multiple DBs or clusters
Model binding	Models bind to a specific connection	Define models per connection instance

Best Practices

Use createConnection() when working with multiple databases or clusters.
Manage event listeners for each connection instance.
Keep model definitions scoped to their respective connections.
Close connections properly when they are no longer needed.

Conclusion

Mongoose's support for multiple connections provides flexibility for complex applications requiring access to multiple MongoDB databases or clusters. Proper management of these connections and their models ensures robust and maintainable data handling.

Multi-mongos Support in Mongoose

2025-09-23T23:49:00.002+05:30

Multi-mongos Support in Mongoose | Rustcode

Multi-mongos Support in Mongoose

In a MongoDB sharded cluster, multiple mongos instances act as query routers to distribute requests across shards. Mongoose and the underlying MongoDB driver support connecting to multiple mongos instances to improve availability and load balancing.

Configuring Multiple mongos Instances

To connect to multiple mongos instances, include all mongos hosts as comma-separated host:port pairs in your connection URI:

mongoose.connect('mongodb://mongos1:27017,mongos2:27017,mongos3:27017/mydb', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

This allows the driver to automatically select and failover between mongos instances as needed.

Benefits of Multi-mongos Support

High Availability: If one mongos instance goes down, the driver can route traffic to others.
Load Balancing: Distributes client requests across multiple mongos routers, improving throughput.
Fault Tolerance: Minimizes single points of failure in sharded deployments.

Connection String Best Practices

Always specify all available mongos hosts in the connection URI.
Use useUnifiedTopology: true for modern server discovery and monitoring.
Monitor application and driver logs to ensure mongos failover works as expected.

Summary Table

Feature	Description	Notes
Multiple mongos hosts	List all mongos routers in connection string	Ensures failover and load balancing
UseUnifiedTopology	Enables modern server monitoring	Recommended default option
Failover support	Automatic switch between mongos instances	Improves reliability

Conclusion

Mongoose's support for multiple mongos instances in a sharded cluster ensures better availability and load balancing for your applications. Properly configuring the connection string and using unified topology options are key to leveraging these benefits effectively.

Replica Set Host Names in Mongoose

2025-09-22T23:47:00.002+05:30

Replica Set Host Names in Mongoose | Rustcode

Understanding Replica Set Host Names in Mongoose

When connecting to a MongoDB replica set with Mongoose, specifying the correct host names of each replica set member is crucial. These host names enable the MongoDB driver to discover the full topology of the replica set, handle failover, and route operations correctly.

Specifying Replica Set Host Names

Your connection URI should include all replica set nodes as comma-separated host:port pairs in the mongodb:// connection string. For example:

mongodb://host1:27017,host2:27017,host3:27017/mydb?replicaSet=myReplicaSet

Including all hosts ensures proper discovery and automatic handling of primary elections and failovers.

DNS and Hostname Resolution

It's important that each host name resolves correctly via DNS or your network configuration. If hostnames do not resolve, the driver may fail to connect or failover properly.

Use fully qualified domain names (FQDNs) if necessary to avoid ambiguity, especially in complex network topologies.

Consistency with Replica Set Configuration

The host names you use in your connection string must match exactly those configured in the replica set's members configuration. Mismatched names can cause connection errors or failover issues.

Example Mongoose Connection

mongoose.connect('mongodb://db1.example.net:27017,db2.example.net:27017,db3.example.net:27017/mydb?replicaSet=rs0', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

Summary Table

Aspect	Details	Notes
Host Names	All replica set members' host:port pairs	Must match replica set configuration exactly
DNS Resolution	Hosts should resolve correctly in network	Use FQDNs if needed
Replica Set Name	Specify `replicaSet` parameter	Mandatory for replica set connections

Best Practices

Ensure all replica set host names are correct and resolvable.
Keep connection string hostnames consistent with replica set configuration.
Use network tools to verify hostname resolution before deployment.
Monitor logs for connectivity or failover issues related to hostname problems.

Conclusion

Correctly specifying replica set host names in Mongoose connection strings is essential for stable and reliable connections to MongoDB replica sets. Paying attention to hostname resolution and consistency with your replica set configuration helps ensure smooth failover and high availability.

Replica Set Connections in Mongoose

2025-09-21T23:37:00.002+05:30

Replica Set Connections in Mongoose | Rustcode

Managing Replica Set Connections in Mongoose

Connecting to a MongoDB replica set in Mongoose requires specifying multiple MongoDB server addresses in your connection string. A replica set provides redundancy and high availability by replicating data across multiple nodes, allowing automatic failover in case of node failures.

Replica Set Connection String

To connect to a replica set, your MongoDB connection URI should include all replica set members as host:port pairs, separated by commas. Additionally, you should specify the replica set name using the replicaSet parameter.

mongoose.connect('mongodb://host1:27017,host2:27017,host3:27017/mydb?replicaSet=myReplicaSet', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

Specifying all members ensures that the driver can discover the full set and handle failover automatically.

Failover and Automatic Reconnection

When the primary node goes down, the replica set elects a new primary automatically. The MongoDB driver and Mongoose will detect this change and reconnect accordingly without requiring manual intervention.

This behavior ensures your application remains available even during node failures.

Replica Set Options

You can customize replica set behavior using connection options such as:

readPreference: Direct reads to primary or secondary nodes.
connectTimeoutMS: Timeout for initial connection attempts.
socketTimeoutMS: Timeout for socket inactivity.

Summary Table

Option	Purpose	Notes
`replicaSet`	The name of the replica set	Required for proper replica set discovery
`readPreference`	Controls how reads are distributed	Defaults to primary
`connectTimeoutMS`	Initial connection timeout	Defaults to 30,000 ms
`socketTimeoutMS`	Timeout for socket inactivity	Defaults to 0 (no timeout)

Best Practices

Always specify all replica set members in the connection string.
Specify the replicaSet name explicitly.
Use useUnifiedTopology: true for modern connection management.
Monitor failover events in application logs to ensure smooth operation.

Conclusion

Mongoose’s support for replica set connections allows you to build fault-tolerant applications by leveraging MongoDB’s high availability features. Proper connection string configuration and understanding failover mechanisms are essential for production-ready apps.

Server Selection in Mongoose

2025-09-20T23:23:00.003+05:30

Server Selection in Mongoose | Rustcode

Understanding Server Selection in Mongoose

When your MongoDB deployment includes a Replica Set or Sharded Cluster, the MongoDB Node.js driver (and by extension, Mongoose) must choose an appropriate server to send operations to. This process is called server selection. Selecting the right server is important for maintaining performance, consistency, and failover handling.

Default Server Selection

By default, Mongoose relies on the MongoDB driver's server selection rules. These rules involve factors like:

Server type (primary or secondary).
Read preference (where read operations are sent).
Topology type (single server, replica set, or sharded cluster).

serverSelectionTimeoutMS

One of the most important options is serverSelectionTimeoutMS. This controls how long the driver will wait when trying to find a suitable server before throwing an error.

mongoose.connect('mongodb://localhost:27017/mydb', {
  serverSelectionTimeoutMS: 5000, // default is 30,000 (30s)
  useNewUrlParser: true,
  useUnifiedTopology: true
});

Tip: Lowering this value can make your app fail fast if MongoDB is unavailable, which is useful in APIs and microservices where latency matters.

Factors Influencing Selection

The driver considers:

Replica set role (primary vs secondary).
Application readPreference (e.g., nearest, primaryPreferred, secondary).
Network latency and server availability.

For example, if you specify readPreference: 'secondary', reads will go to secondaries by default, while writes always go to the primary.

Error Handling

If Mongoose cannot connect to any suitable server within the defined serverSelectionTimeoutMS, it throws a MongooseServerSelectionError.

try {
  await mongoose.connect('mongodb://replicaSetUri', {
    serverSelectionTimeoutMS: 3000
  });
} catch (err) {
  console.error('Server selection error:', err);
}

Summary Table

Option	Purpose	Notes
`serverSelectionTimeoutMS`	Max time to wait for a suitable server	Default: 30,000 ms (30s)
`readPreference`	Where reads are directed (primary/secondary/nearest)	Writes always go to the primary

Best Practices

Lower serverSelectionTimeoutMS in latency-sensitive services.
Use readPreference strategically for load balancing.
Monitor logs for MongooseServerSelectionError to detect failover issues.

Conclusion

Server selection is a critical part of connection management in Mongoose. By tuning serverSelectionTimeoutMS and understanding readPreference, you can make your applications more resilient and performant in replica set and sharded environments.

A Note About keepAlive in Mongoose

2025-09-19T22:20:00.000+05:30

A Note About keepAlive in Mongoose | Rustcode

A Note About keepAlive in Mongoose

Before Mongoose version 5.2.0, enabling the keepAlive option in the MongoDB connection was necessary to initiate TCP keepalive messages. These messages help prevent connections from being closed prematurely due to network inactivity, which can cause errors like "connection closed".

Changes in KeepAlive Behavior

Since Mongoose 5.2.0, keepAlive has been enabled by default, significantly reducing the need for manual setting. Furthermore, as of Mongoose version 7.2.0, the keepAlive option and its associated keepAliveInitialDelay option have been deprecated. It is now recommended to remove these options from your connection configurations to avoid confusion and potential deprecation warnings.

What Does keepAlive Do?

The keepAlive option sends periodic TCP keepalive packets on the socket to maintain the connection to MongoDB even during periods of inactivity. This helps prevent the connection from being closed by network intermediaries like routers or firewalls due to inactivity timeouts.

Modern Mongoose Connection Setup

With recent Mongoose versions, you don’t need to specify keepAlive explicitly. The MongoDB driver and Mongoose handle keepalive internally for you.

mongoose.connect('mongodb://localhost:27017/mydb', {
  useNewUrlParser: true,
  useUnifiedTopology: true
});

Advice for Long Running Applications

Long-running applications, such as web servers or real-time APIs, benefit from persistent connections. The built-in keepalive mechanisms ensure stable connections without the need for manual keepAlive configuration.

Summary Table

Option	Status	Notes
`keepAlive`	Enabled by default since Mongoose 5.2.0	Deprecated as of Mongoose 7.2.0; no manual setting needed
`keepAliveInitialDelay`	Was configurable for initial delay	Also deprecated with keepAlive option

Best Practices

Remove keepAlive and keepAliveInitialDelay from your connection options if you are using Mongoose 7.2.0 or later.
Rely on the MongoDB driver’s default TCP keepalive behavior for connection stability.
Monitor application logs for connection issues unrelated to keepalive configuration.

Conclusion

The keepAlive option was once critical for maintaining MongoDB connections but has been managed automatically by Mongoose in recent versions. Understanding this evolution helps you keep your Node.js apps clean and up-to-date with modern best practices.

How to Merge Two or More Dictionaries in Python

2025-09-19T20:42:00.005+05:30

How to Merge Two or More Dictionaries in Python

Merging dictionaries is a common task when combining data from different sources or updating configuration settings. Python provides multiple ways to merge two or more dictionaries, from the traditional update() method to more modern techniques like dictionary unpacking and the collections.ChainMap. This guide covers these methods with clear examples to help you write clean, efficient Python code.

Table of Content

# Why Merge Dictionaries?
# Syntax & Structure
# Using update() Method
# Merging with Dictionary Unpacking
# Using collections.ChainMap
# Comparison Table: Dictionary Merging Methods
# Useful Tips
# Conclusion

Why Merge Dictionaries?

Combining Data: Efficiently consolidate information from multiple dictionaries.
Updating Configurations: Overlay or augment settings through merging.
Maintainability: Keep data modular by merging when needed.
Handling Defaults: Merge default values with user input or overrides.

Syntax & Structure

The basic idea behind merging dictionaries is to produce a new dictionary or update an existing one such that all key-value pairs from one or more dictionaries are present. When duplicate keys exist, the rightmost dictionary’s value is used.

Using `update()` Method

update() modifies a dictionary in place by adding key-value pairs from another.

Example:

dict1 = {'a': 1, 'b': 2}
dict2 = {'b': 3, 'c': 4}

dict1.update(dict2)
print(dict1)

Output

{'a': 1, 'b': 3, 'c': 4}

Note: dict1 is changed and contains merged data.

Merging with Dictionary Unpacking

Introduced in Python 3.5+, dictionary unpacking creates a new dictionary from multiple sources.

Example:

dict1 = {'a': 1, 'b': 2}
dict2 = {'b': 3, 'c': 4}

merged = {**dict1, **dict2}
print(merged)

Output

{'a': 1, 'b': 3, 'c': 4}

This method does not modify original dictionaries and can merge many dictionaries at once:

dict3 = {'d': 5}
merged_all = {**dict1, **dict2, **dict3}
print(merged_all)

Using `collections.ChainMap`

ChainMap groups multiple dictionaries into a single view without merging data physically. It searches each dictionary in order for keys.

Example:

from collections import ChainMap

dict1 = {'a': 1, 'b': 2}
dict2 = {'b': 3, 'c': 4}

merged_view = ChainMap(dict2, dict1)
print(merged_view['b'])  # Output: 3 (from dict2)
print(merged_view['a'])  # Output: 1 (from dict1)

Note: ChainMap does not create a new dictionary. Changes to original dicts reflect in the view.

Comparison Table: Dictionary Merging Methods

Method	Example	Modifies Original?	Best For
`update()`	`dict1.update(dict2)`	Yes (dict1 updated)	When in-place update is needed
Dictionary unpacking	`{dict1, dict2}`	No (creates new dict)	Creating new merged dictionary; merging many dicts
`collections.ChainMap`	`ChainMap(dict2, dict1)`	No (views dicts)	Non-destructive combined view without copying

Useful Tips

Order matters: In all methods, the last dictionary’s keys overwrite previous ones on duplicates.
Python 3.9+: You can also use the union operator |, e.g., merged = dict1 | dict2.
For deep merges: These methods do shallow merges only. For nested dictionaries, consider recursive merging techniques or third-party libraries.
ChainMap is read-only: Modifying the ChainMap affects only the first dictionary.
Use copy() if needed to avoid modifying originals when using update().

Conclusion

Merging dictionaries in Python is straightforward with multiple available methods depending on your needs. Use update() if you want to modify a dictionary in place, dictionary unpacking {**d1, **d2} to merge into a new dictionary, or ChainMap for a non-destructive, dynamic view. Each approach has its context and benefits, helping you write clean and effective Python code when combining dictionary data.

Connection Events in Mongoose

2025-09-18T22:18:00.005+05:30

Connection Events in Mongoose | Rustcode

Connection Events in Mongoose

Mongoose connections emit a variety of events that you can listen for to monitor and react to the state of your MongoDB connection. These events help you track connectivity, errors, reconnection attempts, and more, allowing your application to handle database availability proactively.

Table of Content

Common Connection Events

connecting: Emitted when Mongoose starts making its initial connection to the MongoDB server.
connected: Emitted when Mongoose successfully makes its first connection or reconnects after losing connectivity.
open: Emitted after connected and onOpen is executed on models.
disconnecting: Emitted when Connection#close() is called to disconnect.
disconnected: Emitted when Mongoose loses connection to MongoDB.
reconnected: Emitted when Mongoose successfully reconnects after losing connection.
close: Emitted after the connection has closed successfully.

Setting Up Event Listeners

Listen to events from the mongoose.connection object:

const db = mongoose.connection;

db.on('connecting', () => console.log('Connecting to MongoDB...'));
db.on('connected', () => console.log('Connected to MongoDB!'));
db.on('open', () => console.log('Connection open'));
db.on('disconnecting', () => console.log('Disconnecting from MongoDB'));
db.on('disconnected', () => console.log('Disconnected from MongoDB'));
db.on('reconnected', () => console.log('Reconnected to MongoDB'));
db.on('close', () => console.log('Connection closed'));
db.on('error', err => console.error('Connection error:', err));

Handling Connection Errors

Use the error event to handle and log any errors that occur during the connection lifecycle.

Monitoring Reconnection

Mongoose automatically tries to reconnect when connectivity is lost. Use the reconnected event to confirm when your application regains access to the database.

Example Code

const mongoose = require('mongoose');

mongoose.connect('mongodb://localhost:27017/mydb', { useNewUrlParser: true, useUnifiedTopology: true });

const db = mongoose.connection;

db.on('connecting', () => console.log('Connecting to MongoDB...'));
db.on('connected', () => console.log('Connected to MongoDB!'));
db.on('open', () => console.log('Connection open'));
db.on('disconnecting', () => console.log('Disconnecting from MongoDB'));
db.on('disconnected', () => console.log('Disconnected from MongoDB'));
db.on('reconnected', () => console.log('Reconnected to MongoDB'));
db.on('close', () => console.log('Connection closed'));
db.on('error', err => console.error('Connection error:', err));

Best Practices

Use connection events to provide detailed logging and monitoring.
Implement retry and fallback logic based on connection state.
Gracefully handle disconnections and resource cleanup.
Notify users or administrators on prolonged connectivity issues.

Conclusion

Connection events in Mongoose offer powerful hooks to monitor MongoDB connectivity status and handle events appropriately. Leveraging these events ensures your Node.js applications maintain high availability and resilience against network disruptions.

How to Sort a List of Dictionaries by Key in Python

2025-09-18T20:34:00.010+05:30

How to Sort a List of Dictionaries by Key in Python

Sorting a list of dictionaries by a specific key is a common task in Python data processing and handling collections of complex data. Python’s built-in sorted() function and the list sort() method, combined with the use of a key parameter often defined as a lambda function, make it easy to customize sorting behavior. In this article, you’ll learn how to sort lists of dictionaries by their keys with examples, including ascending and descending order sorts, and multi-level sorting.

Table of Content

# Why Sort a List of Dictionaries?
# Syntax & Structure
# Basic Sorting Examples
# Sorting in Descending Order
# Sorting by Multiple Keys
# Comparison Table: sorted() vs list.sort()
# Useful Tips
# Conclusion

Why Sort a List of Dictionaries?

Organize Data: Make data easier to read or process in a specific order.
Search Optimization: Sorted data can speed up searching algorithms.
Display Purposes: Present information ordered by relevant fields like names, dates, or scores.
Data Analysis: Prepare structured data for reporting or further statistical operations.

Syntax & Structure

# Using sorted() function (returns new sorted list)
sorted_list = sorted(your_list, key=lambda d: d['key_name'])

# Using list's sort() method (sorts in place)
your_list.sort(key=lambda d: d['key_name'])

your_list: The list of dictionaries to be sorted.
key: A function that extracts the sorting key from each dictionary, commonly a lambda.
'key_name': The dictionary key to sort by.

Basic Sorting Examples

1. Sort by a Single Key in Ascending Order

people = [
    {"name": "Alice", "age": 30},
    {"name": "Bob", "age": 25},
    {"name": "Charlie", "age": 35}
]

sorted_people = sorted(people, key=lambda x: x["age"])
print(sorted_people)

Output


[
  {'name': 'Bob', 'age': 25},
  {'name': 'Alice', 'age': 30},
  {'name': 'Charlie', 'age': 35}
]

2. Sorting In-Place Using `list.sort()`

people.sort(key=lambda x: x["name"])
print(people)

Output


[
  {'name': 'Alice', 'age': 30},
  {'name': 'Bob', 'age': 25},
  {'name': 'Charlie', 'age': 35}
]

Sorting in Descending Order

Use the `reverse=True` Parameter

sorted_people_desc = sorted(people, key=lambda x: x["age"], reverse=True)
print(sorted_people_desc)

Output


[
  {'name': 'Charlie', 'age': 35},
  {'name': 'Alice', 'age': 30},
  {'name': 'Bob', 'age': 25}
]

Sorting by Multiple Keys

When dictionaries have multiple relevant keys, you can sort by a tuple of keys for hierarchical sorting.

Example: Sort by Age, then by Name

people = [
    {"name": "Alice", "age": 30},
    {"name": "Bob", "age": 30},
    {"name": "Charlie", "age": 25}
]

sorted_people = sorted(people, key=lambda x: (x["age"], x["name"]))
print(sorted_people)

Output


[
  {'name': 'Charlie', 'age': 25},
  {'name': 'Alice', 'age': 30},
  {'name': 'Bob', 'age': 30}
]

Comparison Table: `sorted()` vs `list.sort()`

Feature	`sorted()`	`list.sort()`	Best For
Return Value	Returns a new sorted list	Sorts the list in place and returns `None`	Preserving original vs modifying data
Usability	Works with any iterable (not just lists)	Only works on lists	General sorting vs performance on list
Syntax	`sorted_list = sorted(lst, key=func)`	`lst.sort(key=func)`	Preference and use case dependent
Performance	Slower due to creating new list	Faster since it sorts in place	When memory < performance and vice versa

Useful Tips

Use key=lambda x: x['key_name'] to specify the sorting key elegantly.
Remember reverse=True to order descendingly.
For complex sorting, define a normal function instead of a lambda.
Check keys exist: Handle missing keys or provide defaults to avoid KeyError.
Sorting is case sensitive: Use str.lower in key if needed (e.g., key=lambda d: d['name'].lower()).
For heavy sorting: Consider using operator.itemgetter() for potentially better performance.

Conclusion

Sorting a list of dictionaries by key in Python is straightforward using the sorted() function or the in-place list sort() method with a key parameter. Mastering the use of lambda functions and multi-key sorting will allow you to efficiently organize and manipulate complex data structures with ease.

Connection String Options in Mongoose

2025-09-17T22:10:00.001+05:30

Connection String Options in Mongoose | Rustcode

Connection String Options in Mongoose

Mongoose allows you to specify connection options directly in the MongoDB connection string URI as parameters appended in the query string. These options configure crucial aspects such as authentication, timeouts, retry policies, and security.

Table of Content

What Are Connection String Options?

Connection string options are key-value pairs appended to the MongoDB URI after a question mark (?). They configure driver-level behaviors and MongoDB server settings such as read preferences, replica sets, and authentication sources.

Common Connection String Options

authSource: Specify the database to authenticate against.
retryWrites: Enable or disable retryable writes.
ssl: Enable SSL connections.
replicaSet: Name of the replica set to connect to.
readPreference: Controls how reads are routed to replica set members.
connectTimeoutMS: Time in milliseconds to attempt connection before timing out.
socketTimeoutMS: How long to wait for responses before timing out.

Advantages and Disadvantages

Advantages: Simplifies configuration; everything can be stored in one URL.
Disadvantages: Can become unwieldy and unreadable for complex configurations; some Mongoose-specific options cannot be set here.

How to Set Connection String Options

Include options as query parameters in your MongoDB URI. For example:

mongodb://localhost:27017/mydb?ssl=true&authSource=admin&retryWrites=true

Then pass this URI to mongoose.connect():

mongoose.connect('mongodb://localhost:27017/mydb?ssl=true&authSource=admin&retryWrites=true');

Tips and Best Practices

Use connection string options for settings closely related to the host, auth, or cluster.
For options that vary by environment, prefer passing them as options objects instead of in the URI.
Do not include sensitive options like bufferCommands in the URI as they are Mongoose-specific and not recognized via connection string.
Use URI encoding for special characters in user names or passwords.

Example Connection String with Options

mongoose.connect(
  'mongodb://localhost:27017/mydb?ssl=true&authSource=admin&retryWrites=true', 
  { useNewUrlParser: true, useUnifiedTopology: true }
);

Conclusion

Connection string options in Mongoose provide a convenient way to configure many essential aspects of your MongoDB connections directly in the URI. While they simplify setup, careful use combined with options passed programmatically helps achieve optimal configuration and maintainability.

How to Create and Manipulate Sets in Python

2025-09-17T20:24:00.006+05:30

How to Create and Manipulate Sets in Python

Sets in Python are unordered collections of unique elements, making them perfect for membership testing, removing duplicates, and performing mathematical set operations like union, intersection, and difference. This article will help you understand how to create sets, manipulate them effectively, and use their powerful features in your Python programs.

Why Use Sets?

Uniqueness: Automatically removes duplicate elements.
Fast Membership Testing: Checking if an element is in a set is highly efficient.
Mathematical Operations: Easily perform union, intersection, difference, and symmetric difference.
Unordered Collections: Store elements without caring about order.

Syntax & Structure

# Creating a set from a list
my_set = set([1, 2, 3, 4])

# Creating a set with curly braces (preferred)
my_set = {1, 2, 3, 4}

# Empty set (note: {} creates an empty dict, use set() instead)
empty_set = set()

{} is used to create sets with elements inside.
To create an empty set, use set(), not {} because that creates an empty dictionary.
Sets automatically remove duplicate values at creation.

Basic Set Examples

1. Create and Print a Set

fruits = {"apple", "banana", "orange", "apple"}
print(fruits)

Output (note duplicate removed)

{'apple', 'banana', 'orange'}

2. Convert List to Set to Remove Duplicates

nums = [1, 2, 2, 3, 4, 4, 4]
unique_nums = set(nums)
print(unique_nums)

Output

{1, 2, 3, 4}

3. Check Membership

fruits = {"apple", "banana", "orange"}
print("banana" in fruits)  # True
print("grape" in fruits)   # False

Output

True
False

Common Set Operations

Union: Combine Sets (all unique elements)

a = {1, 2, 3}
b = {3, 4, 5}
print(a | b)        # or a.union(b)

Output

{1, 2, 3, 4, 5}

Intersection: Elements in Both Sets

print(a & b)        # or a.intersection(b)

Output

{3}

Difference: Elements in a but not in b

print(a - b)        # or a.difference(b)

Output

{1, 2}

Symmetric Difference: Elements in either a or b but not both

print(a ^ b)        # or a.symmetric_difference(b)

Output

{1, 2, 4, 5}

Modifying Sets: Add, Remove, Update

Add an Element

s = {1, 2, 3}
s.add(4)
print(s)

Output

{1, 2, 3, 4}

Remove an Element (raises KeyError if not found)

s.remove(3)
print(s)

Output

{1, 2, 4}

Discard an Element (no error if not found)

s.discard(5)  # does nothing since 5 not in set
print(s)

Output

{1, 2, 4}

Pop an Arbitrary Element

elem = s.pop()
print("Popped:", elem)
print(s)

Output (example)

Popped: 1
{2, 4}

Update with Another Set or Iterable

s.update({5, 6})
print(s)

Output

{2, 4, 5, 6}

Comparison Table: Sets vs Lists

Feature	`Set`	`List`	Best For
Uniqueness	Stores unique elements only	Allows duplicates	Removing duplicates, unique collections
Order	Unordered, no indexing	Ordered, accessible by index	Order matters vs order not important
Membership Testing	Very fast (hash-based)	Slower (linear scan)	Fast lookup of existence
Mutability	Mutable (add/remove elements)	Mutable (add/remove elements)	Both support mutation
Supports Duplicates	No	Yes	Control duplicates or not

Useful Tips

Use set() to convert existing iterables into unique collections.
Remember: Sets are unordered — you can't rely on element order.
For immutable sets: Use frozenset() which cannot be modified after creation.
Use set operations: They are very helpful for data analysis, deduplication, and membership checks.
Do not use sets if order or duplicates matter: Use lists or tuples instead.
Be aware of the error behavior: remove() raises an error if item not found; discard() does not.

Conclusion

Python sets provide a powerful and efficient way to handle collections of unique items and perform mathematical set operations. They excel when order is unimportant but uniqueness and fast membership testing are essential. Knowing how to create, modify, and utilize sets will make your Python programs cleaner, faster, and more expressive when dealing with such requirements.

serverSelectionTimeoutMS in Mongoose

2025-09-16T22:07:00.001+05:30

serverSelectionTimeoutMS in Mongoose | Rustcode

serverSelectionTimeoutMS in Mongoose

The serverSelectionTimeoutMS option in Mongoose controls how long the MongoDB Node.js driver will try to select a suitable server for operations before throwing an error. It affects both the initial connection attempt and subsequent database operations. Understanding and configuring this timeout is crucial for managing connection failures and improving responsiveness.

Table of Content

What is serverSelectionTimeoutMS?

The serverSelectionTimeoutMS setting tells the MongoDB driver how long it should keep trying to select a server for read or write operations before it declares failure and returns an error.

Default Value

By default, serverSelectionTimeoutMS is set to 30000 milliseconds (30 seconds). This means Mongoose will attempt server selection for up to 30 seconds before timing out.

Impact on Connection and Operations

This timeout applies to:

The initial connection attempt via mongoose.connect().
Any operations that require sending requests to the database like queries, inserts, updates, etc.

If the driver cannot find an available server within this time, it throws a MongoTimeoutError.

How to Set the Timeout

Set this option in the connection options object passed to mongoose.connect():

mongoose.connect('mongodb://localhost:27017/mydb', {
  serverSelectionTimeoutMS: 5000 // 5 seconds timeout
});

Error Handling with Timeout

If the timeout is reached, Mongoose throws a MongoTimeoutError which includes a reason property describing why selection failed (e.g., DNS error, authentication failure).

mongoose.connect('mongodb://badhost:27017/mydb', {
  serverSelectionTimeoutMS: 5000
}).catch(err => console.error('Connection failed:', err.reason));

Best Practices

Adjust serverSelectionTimeoutMS depending on network reliability and environment.
For serverless environments (like AWS Lambda), consider reducing timeout for faster failure detection.
For replica sets, use a higher timeout to accommodate elections and failovers.
Always handle MongoTimeoutError in your application for graceful failures.

Example Usage

const mongoose = require('mongoose');

async function connectWithTimeout() {
  try {
    await mongoose.connect('mongodb://localhost:27017/mydb', {
      serverSelectionTimeoutMS: 5000
    });
    console.log('Connected to MongoDB');
  } catch (err) {
    if (err instanceof mongoose.MongoTimeoutError) {
      console.error('Server selection timed out:', err.reason);
    } else {
      console.error('Other connection error:', err);
    }
  }
}

connectWithTimeout();

Conclusion

The serverSelectionTimeoutMS option is crucial to controlling MongoDB server selection retry behavior in Mongoose. Properly tuning and handling this timeout helps improve application responsiveness and robustness when MongoDB servers are unavailable or slow to respond.

How to Work with Tuples in Python: Immutable Sequences Explained

2025-09-16T20:17:00.009+05:30

How to Work with Tuples in Python(Immutable Sequences Explained)

Tuples are one of Python’s fundamental built-in data structures. They are ordered, immutable sequences often used to group related data together. Unlike lists, tuples cannot be changed after creation, making them useful for fixed collections of values where immutability is desired. This article explains how to create, access, and work with tuples effectively in your Python code.

Table of Content

# Why Use Tuples?
# Syntax & Structure
# Basic Tuple Examples
# Common Tuple Operations
# Tuple Packing and Unpacking
# Comparison Table: Tuples vs Lists
# Useful Tips
# Conclusion

Why Use Tuples?

Immutability: Tuples are immutable, making them safe from accidental changes.
Performance: Tuples have smaller memory overhead and can be faster than lists.
Data Integrity: Use tuples to represent data that should not be modified, such as coordinates or fixed configurations.
Dictionary Keys: Tuples can be used as dictionary keys because they are hashable.

Syntax & Structure

# Creating a tuple
my_tuple = (1, 2, 3)

# Tuple without parentheses (also valid)
another_tuple = 4, 5, 6

# Single element tuple (note the comma)
single_element = (7,)

# Empty tuple
empty_tuple = ()

() - Parentheses enclose tuple elements (optional if unambiguous).
Tuples are defined by commas separating values; parentheses help readability.
Single element tuples require a trailing comma to differentiate from a simple value.

Basic Tuple Examples

1. Creating and Printing a Tuple

point = (10, 20)
print(point)

Output

(10, 20)

2. Single Element Tuple

single = ("hello",)
print(single)
print(type(single))

Output

('hello',)
<class 'tuple'>

3. Tuple Without Parentheses

colors = "red", "green", "blue"
print(colors)

Output

('red', 'green', 'blue')

Common Tuple Operations

Accessing Elements

tpl = (100, 200, 300)
print(tpl[0])   # First element
print(tpl[-1])  # Last element

Output

100
300

Slicing Tuples

tpl = (10, 20, 30, 40, 50)
print(tpl[1:4])

Output

(20, 30, 40)

Tuple Length, Concatenation, and Repetition

t1 = (1, 2, 3)
t2 = (4, 5)

print(len(t1))           # Length
print(t1 + t2)           # Concatenate
print(t1 * 2)            # Repeat

Output

3
(1, 2, 3, 4, 5)
(1, 2, 3, 1, 2, 3)

Checking Membership

colors = ("red", "green", "blue")
print("green" in colors)
print("yellow" not in colors)

Output

True
True

Counting and Finding Index

numbers = (1, 2, 3, 2, 2, 4)
print(numbers.count(2))      # Count occurrences
print(numbers.index(3))      # Find index of first occurrence

Output

3
2

Tuple Packing and Unpacking

Packing Multiple Values into a Tuple

packed = 1, 2, 3
print(packed)

Unpacking Tuple Elements into Variables

point = (10, 20)
x, y = point
print("x =", x)
print("y =", y)

Output

x = 10
y = 20

Unpacking with Ignore Variable

data = (1, 2, 3, 4)
a, b, *rest = data
print(a, b)
print(rest)

Output

1 2
[3, 4]

Comparison Table: Tuples vs Lists

Feature	`Tuple`	`List`	Best For
Mutability	Immutable	Mutable	Fixed vs dynamic data
Syntax	`(1, 2, 3)`	`[1, 2, 3]`	Immutability with tuples, flexibility with lists
Performance	Faster, smaller memory footprint	Slower due to mutability overhead	When speed and safety matter
Use as dictionary key	Yes (hashable)	No (unhashable)	Keys or fixed groups of data
Support for mutation methods (append, remove, etc.)	No	Yes	Dynamic collections

Useful Tips

Use tuples for fixed collections: When data should remain constant, use tuples for safety.
Remember the comma in single element tuples: Without the comma, it’s just a value in parentheses.
Unpack tuples carefully: Ensure the number of variables matches the number of items.
Tuples can contain mutable objects: Immutability refers to the tuple’s structure, not the objects inside.
Use tuples as dictionary keys when you need compound keys.
For large sequences requiring modification, prefer lists.

Conclusion

Tuples provide a lightweight, immutable sequence type for grouping related data in Python. They are perfect for fixed-size, read-only collections, offer performance benefits, and fit naturally in many scenarios like multiple-return values and dictionary keys. Understanding tuple creation, access, and unpacking will strengthen your Python programming and give you more flexible tools for data handling.

Connection Options in Mongoose

2025-09-15T22:04:00.001+05:30

Connection Options in Mongoose | Rustcode

Connection Options in Mongoose

Mongoose's connect() method supports various options to customize the connection behavior to MongoDB. These options help manage authentication, timeout handling, connection pooling, retry logic, and other aspects critical to a stable and performant application.

Table of Content

Authentication Options

user and pass: Username and password for authentication.
authSource: Specifies the authentication database.
authMechanism: The authentication mechanism, e.g., SCRAM-SHA-1, SCRAM-SHA-256, etc.

Timeout Options

connectTimeoutMS: How long to wait for a connection before timing out.
socketTimeoutMS: How long the socket should stay idle before timing out.
serverSelectionTimeoutMS: Time to wait for server selection before throwing an error.

Connection Pooling Options

maxPoolSize: Maximum number of sockets in the connection pool.
minPoolSize: Minimum number of sockets in the connection pool.
waitQueueTimeoutMS: Maximum time to wait for a connection from the pool.

Retry and Buffering Options

retryWrites: Enables retryable writes.
bufferCommands: Enable or disable Mongoose command buffering.
bufferMaxEntries: Maximum number of operations Mongoose buffers.

SSL and Security Options

ssl: Enable SSL connection.
sslCA: Certificate authority for SSL.
sslCert and sslKey: Client certificate and key.

Example Connection with Options

const mongoose = require('mongoose');

const options = {
  user: 'myUser',
  pass: 'myPassword',
  authSource: 'admin',
  authMechanism: 'SCRAM-SHA-256',
  connectTimeoutMS: 10000,
  socketTimeoutMS: 45000,
  maxPoolSize: 10,
  retryWrites: true,
  useNewUrlParser: true,
  useUnifiedTopology: true
};

mongoose.connect('mongodb://localhost:27017/mydb', options)
  .then(() => console.log('Connected to MongoDB'))
  .catch(err => console.error('Connection error:', err));

Best Practices

Specify authentication options explicitly to avoid connection failures.
Adjust timeout options based on network conditions and application needs.
Tune connection pool size to balance resource usage and throughput.
Enable retryWrites for resilient write operations.
Use SSL options when connecting to secured MongoDB instances.

Conclusion

Mongoose connection options provide powerful controls over how your application connects to MongoDB, affecting authentication, performance, security, and reliability. Understanding and configuring these options will help you build robust Node.js applications with efficient and secure database connections.