Distributed System 思维导图模板_ProcessOn思维导图、流程图

DS Issues

Concurrency

Tackling

Semaphore

Making access sequential

Multiple clients can access the same resource at the same time

Issure

Components executed in concurrent processes read and update share resources

No global clock

Heterogeneity

Issue

Inconsistency in software, hardware, operating system and programming language, etc

Tackling

Use standard protocols

Use agreed upon formats or data types

Adhering to an agreed upon api

Use middleware

Java RMI (JRMP procotol)

RPC (http protocol)

Openness

Issue

Extension and integration with new components

Tackling

Publishing key interfaces

Provision a uniform communication mechanism

Security

Issue

Network traffic is not secure concerning data confidentiality, data integrity and availability

Tackling

Checksum (integrity control)

MD5

SHA1

Authorization

(access control lists, certificate)

Authentication

(passwords, public key authentication)

Encryption

RSA

Scalability

Issue

Concerns whether the system remains effective when there's rise of demand(resources and users)

Tackling

Avoiding Performance bottlenecks: Decentralized algorithms should be used to avoid performance bottlenecks.

Resources should not run out (e.g. IP addresses 32bits to 128bits)

Controlling the performance loss

Controlling cost of physical resources (not grow too much as the system scales)

Failure Handling

Tackling

Redundancy

using redundant components (e.g. multiple routes between to routers, database replication)

Recovery

it can recover the original state after a crash

Tolerating

failure is reported back to the user (e.g. web server not being available)

Masking

(e.g. message retransmission, one server fails, try another)

Detecting

Issue

Concerns whether the system continue to function when some of its nodes fail

Transparency

Concealment from the user and the application programmer of the separation of components in a distributed system

Model

Communication Paradigms

Interprocess communication

Remote invocation

Indirect communication

group communication, publish-subscribe systems, message queues, tuple spaces, distributed shared memory

Architectural Styles

Client-server

Clients invoke services in servers and results are returned

Clients initiate connections, servers listen for/receive connections

Poor scalability

Peer-to-peer

Each process in the systems plays a similar role interacting cooperatively as peers (playing the roles of client and server simultaneously)

Bit torrent

Features

Services provided by multiple servers

A service may be separated into multiple processes running on multiple hosts on the system network.

Partition the set of objects on which the service is based and distribute those objects between themselves

(e.g. Sun Network Information Service (NIS))

Proxy servers and caches

Cache: A store of recently used data objects for easier access

Increase the availability and performance of the service by reduce duplicate loading of resources over network

Proxy Server: A server act as broker for requests and access distribution

Improve security and privacy protection, load balance

Mobile Code and Agents

Mobile code

Mobile code is sent from one computer to another to run at the destination

Examples

Java Applet

Javascript

Flash

Mobile Agents

A running program (code + data) that travels from one computer to another in a network carrying out a task on someone's behalf

http://www.codeproject.com/Articles/11064/Mobile-Agents-Software-on-the-move

Examples

IBM Aglet

Network Computers

e.g. Cloud machine with package managers

Thin Clients

Business logic not downloaded but executed on server

Move complexity away from the end-user device

Layering

Performance of communication channels

Latency

Bandwidth

Jitter

Models

Physical Model

A representation of underlying hardware elements and composition of a system.

Architecture Model

Components and their roles, how they interact with each other and underlying system.

Fundamental Model

Defines the non-functional aspects of the distributed system such as reliability, security, and performance.

Interaction Model

Concerned with the performance of processes and communication channels and the absence of a global clock

Failure Model

Classifies the failures of processes and basic communication channels in a distributed system

Security Model

Identifies the possible threats to processes and communication channels

DS Characterization

Failure Transparency

the extent to which errors and subsequent recoveries of hosts and services within the system are invisible to users and applications

Byzantine Tolerance

failed node can generate arbitrary data, pretending to be a correct one, which makes fault tolerance utterly difficult.

Fault: Any fault presenting different symptoms to different observers

Failure: The loss of a system service due to a Byzantine fault in systems that require consensus (a node is trying to do sth arbitrarily)

http://blog.sina.com.cn/s/blog_5c4b68750100v151.html

Middleware

Middleware is a software layer between the distributed application and the operating systems which hides the underlying heterogeneity

Examples

1. JVM

2. MessageQueue

3. Proxy Server

4. Distributed application integration(e.g. Weblogic)

DS Issues

Concurrency

Tackling

Making access sequential

Multiple clients can access the same resource at the same time

Semaphore

Issure

Components executed in concurrent processes read and update share resources

No global clock

Heterogeneity

Issue

Inconsistency in software, hardware, operating system and programming language, etc

Tackling

Use agreed upon formats or data types

Adhering to an agreed upon api

Use middleware

Java RMI (JRMP procotol)

RPC (http protocol)

Use standard protocols

Openness

Issue

Extension and integration with new components

Tackling

Publishing key interfaces

Provision a uniform communication mechanism

Security

Issue

Network traffic is not secure concerning data confidentiality, data integrity and availability

Tackling

Authorization

(access control lists, certificate)

Authentication

(passwords, public key authentication)

Encryption

RSA

Checksum (integrity control)

MD5

SHA1

Scalability

Issue

Concerns whether the system remains effective when there's rise of demand(resources and users)

Tackling

Avoiding Performance bottlenecks: Decentralized algorithms should be used to avoid performance bottlenecks.

Resources should not run out (e.g. IP addresses 32bits to 128bits)

Controlling cost of physical resources (not grow too much as the system scales)

Controlling the performance loss

Failure Handling

Tackling

Recovery

it can recover the original state after a crash

Tolerating

failure is reported back to the user (e.g. web server not being available)

Masking

(e.g. message retransmission, one server fails, try another)

Detecting

Redundancy

using redundant components (e.g. multiple routes between to routers, database replication)

Issue

Concerns whether the system continue to function when some of its nodes fail

Transparency

Concealment from the user and the application programmer of the separation of components in a distributed system

Omission Failure

Omission failures either due to process crash or communication link failures are detected via timeouts.

Web Services

Web services paradigm allows programs (not just browsers) to be clients for web programs

Data between the server and the client is often passed using Extensible Markup Language (XML)/JSON.

SOAP protocol (Simple Object Access protocol) allows client to invoke web services.

REST Protocol, similar to SOAP but not as secure as SOAP

RMI and RPC

JAVA RMI

Failure Tolerance Measures

Mechanism to filter duplicates

Strategy for results retransmission

Strategy to retry request message

Semantics

Maybe invocation semantics

omission failure, crash failures

At-least-once invocation semantics

crash failures, arbitrary failures

At-most-once

complete fault tolerance

Java RMI and CORBA support at-most-once invocation semantic.

CORBA allows maybe semantics for calls that do not return values.

Sun RPC supports at-least-once semantic.

Common

Remote Object

An object that can receive remote invocations is called a remote object.

Remote Object Reference

A unique identifier that can be used throughout the distributed system for identifying an object

Remote Interface

A remote interface defines the methods that can be invoked by external processes. Remote objects implement the remote interface.

Garbage collection

Is achieved through reference counting.

Proxy/Stub

Locates at client side, responsible for arguments marshaling and un-marshaling , as well as reference

Marshalling the reference of the target object, its own method id and the arguments and forwarding them to the communication module.

Unmarshalling the results and forwarding them to the invoking object

Dispatcher

There is one dispatcher for each remote object class.

Is responsible for mapping to an appropriate method in the skeleton based on the method ID.

Skeleton

Locates at server side, responsible for code structure and execution

Marshalling request from client to servant, Un-marshalling results from servant to client, and forwarding them to corresponding destination

RPC

Transparancy

Access Transparency

Clients should be unaware of the distribution of the files.

The files could be present on a totally different set of servers which are physically distant apart

a single set of operations should be provided to access these remote as well as the local files

Location Transparency

Files or groups of files may be relocated without changing their pathnames.

location transparent name contains no information about the named object’s physical location.

e.g. access resources by universal api

Concurrency Transparency

Users and Applications should be able to access shared data or objects without interference between each other.

Replication Transparency

replicate the data at two or more sites for more reliability

The client generally should not be aware that a replicated copy of the data exists

Failure Transparency

concealment of faults

User can complete their tasks despite the failure of software/hardware component

Mobility Transparency

Load balance

Performance Transparency

Allows the system to be reconfigured to improve the performance as the load varies

Scaling Transparency

A system should be able to grow without affecting application running.

Indirect Communication

Event

Observer=Publisher

Forwarding

Sending notifications to all subscribers who have expressed interest in the event

Filtering of notifications

When notifications are based on particular conditions the observer filters the event of interest

Patterns of events

Correlating events from multiple observers to understand patterns.

Notification mailboxes

In some cases the notification can have to delayed until the subscriber is ready to receive this information

Publish-Subscriber System

Type

Channel Based

Publishers publish to named channels and subscribers subscribe to all events on a named channel.

Type Based

Subscribers register interest in types of events and notifications occur when particular types of events occur.

Topic Based

Subscribers register interest in particular topics and notifications occur when any information related to the topic arrives.

Content Based

Subscribers can specify interest in particular values or ranges of values for multiple attributes(e.g. with favorite tags).

Group

Typical APIs

Create Group

Join Group

Leave Group

Remove Failed member

Multi-cast

Ordering

FIFO

Preserving order on the sender side

causal ordering

A message HAPPENS earlier would be preserved in the delivery at all processes

total ordering

A message DELIVERED earlier would be preserved in the delivery at all processes

Group address changed

Openess

Close group, only internal comm

Open group, both internal and external comms

Memory Sharing

Tuple Spaces

Distributed Share Memory

Message Queue

send

blocking receive

a consumer waits for at least one message on a queue then returns

non-blcoking receive

polling, a consumer will check and get a message if there, otherwise it returns without a message

notify

https://www.youtube.com/watch?v=szqdtIEgTR4

Definition

Communication between entities in a distributed system through an intermediary

with no direct coupling between the sender and the receiver(s).

Coupling

Space uncoupling

sender does not know or need to know the identity of the receiver(s)

Time uncoupling

sendder and receiver can have independent lifetimes, they do not need to exist at the same time

Even asynchronous processes may have same lifetime, so time uncoupling is an significant feature of IC