5. ATAM Phase 0: Evaluation

 

Purpose: Partnership & Preparation: Usually present the ATAM to a small group. Get the Business Drivers.

Step 1: Purpose to ensure that the client understands the mechanics of the evaluation method; make sure the client understands the CBA of an architecture evaluation. Record questions for possible FAQ list. Consultants may write up work plans. 75 man days for evaluation team effort– duration best case is 3 weeks.

Step 2: Initial description of the candidate system. Client provides existing documentation describing the system. Client conveys main architectural drivers e.g. business goals, requirements, constraints etc.

Client and evaluation organization agree on necessary architecture documentation – “3 main views”.

NDA – for evaluation team is done at this step.

Evaluators record general business goals, quality attributes, architectural constraints and list of architecture documentation to be delivered to the evaluation team.

Step 3: Go/No-Go decision with respect to conducting ATAM.

Evaluation organization representatives understand the state of the architecture well enough to make a decision and ensure that the candidate system is ready for evaluation.

Evaluation team takes a look at the context drawing and multiple views of the system (e.g. run time etc.)

The list of named participants and their roles with respect to the system must be provided.

Step 4: SOW presentation and negotiation.

Step 5: Form core evaluation team. Aim for 4-6 evaluators.

Modifiability – coupling, encapsulation, contract based interactions, cohesion etc.

Step 6: Conduct evaluation team kick-off meeting.

Team Leader: establishes the time and place for the meeting.

Stickies on a board that can be grouped by risk themes are helpful…or use “Mind Maps”.

Tools like “Enterprise Architect” are used for evaluation in some companies.

Step 7: Prepare and Plan for Phase 1.

Review the purpose of the ATAM phases with the client.

Confirm the time and place for the evaluation for the client to present the system architecture & business goals, architect to present the system architecture and arrange for supplies.

Step 8: Preliminary review of the system’s software architecture.

Hold a brief post-mortem.

4. ATAM Phases Overview

 

There are 4 phases of the ATAM evaluation: Phase 0-3.

Phase 0: Partnership & Preparation

· Usually present the ATAM to a small group. Get the Business Drivers.

Phase 1: Initial Evaluation: Step 1-6

· Steps 1-5: We don’t pre-judge here. Just gather information and focus on the pros.

· Step 6: This is still phase 1. Ask questions about the architectural decisions, and do they map back to business drivers?

Phase 2: Complete Evaluation: Step 7-9

· Step 7: (Brainstorm & Prioritize) – Phase 2: Show Phase 1 scenarios, you recap.

· Step 8: Analyze Architectural Approaches: You have more stakeholders.

· Step 9: Report out

Phase 3: Follow-up

3. The Benefits of ATAM Evaluations

 The following are the benefits of Architecture Tradeoff Analysis Methodology (ATAM) -

· Clarified Quality Attribute Requirements

· Improved Architecture documentation

· Documented basis for architectural decisions

· Identify risks early in the life cycle

· Increased communication among stakeholder

· The results are dramatically improved software and solution architectures.

2. ATAM Conceptual Model

The Architecture Tradeoff Analysis Method (ATAM) is a facilitated, stakeholder-driven evaluation process designed to surface risks, non-risks, sensitivity points, and trade-offs in an architecture. Its value lies in creating a structured conversation that aligns technical decisions with business drivers and quality attributes.

Core Concepts

Sensitivity Points

Definition: A property of one or more components that is critical to achieving a specific quality attribute response.

Example: Queue depth is a sensitivity point. Adjusting it directly influences scalability and throughput.

Trade-offs

Definition: A property that simultaneously affects multiple quality attributes, often requiring explicit prioritization.

Example: Persistent vs. non-persistent queues impact durability, availability, and throughput—forcing a conscious trade-off among them.

Conceptual Flow

The ATAM process decouples business drivers and scenarios from the architectural plan and decisions, ensuring a traceable line of reasoning.

A more accurate flow in practice is:

Architectural Plan / Presentation

→ Architectural Approaches (candidate solutions and patterns)

→ Quality Attribute Requirements (QARs) tied to those approaches

→ Architectural Decisions made explicit and evaluated against QARs

Practitioner Advice

Phase Separation: Never run Phase 1 (eliciting drivers and scenarios) and Phase 2 (detailed analysis and trade-offs) in the same week.

Allow 1–2 weeks between phases for participants to reflect, refine scenarios, and prepare.

This gap leads to sharper analysis and more meaningful stakeholder engagement.

Why It Matters for Architects

Brings clarity and rigor to architectural decisions by making implicit trade-offs explicit.

Provides a shared vocabulary for stakeholders and technologists to discuss quality attributes (scalability, availability, performance, etc.).

Ensures that architecture is justified by business drivers, not just technical preference.

See Conceptual Model here: http://www.sei.cmu.edu/architecture/tools/evaluate/atam.cfm

Understanding Architectures with Pictures

 

“Pictures speak a 1024 words.” – this is a quote I used a lot for the past 10 years or so. Why? Well this is because architectures need to be visualized. Bredemeyer Consulting’s Visual Architecting Process, SEI’s SAPP, RUP’s UML, Zachman, TOGAF etc. all dwell on visualizing abstractions. But how do you translate that to real projects, and especially those that claim to be agile and misunderstand that to mean no design and no plans? As one of the many signatories of the agile manifesto it is clear to me that those projects that do not know what the architecture is can not deliver software in a timely manner with good quality attributes.

Software Architecture should be represented by a set of views that support its analysis. Usually the following views are most often used:

Advice: At least 3-views recommended by SEI:

1. Module View

2. Component View

3. Deployment View

Plus a sequence diagram can be added as the forth.

Multiple views of a software architecture allow it to be understandable without any confusion by the entire team and its stakeholders.

Architect with security in mind as a first thought

So if you’re doing a solution architecture review, make sure you first look at the security design of the system including authentication, digital signatures, secret key cryptography, public key cryptography, authorization, and non-repudiation from the perspective of a digital firm. Authentication and authorization are the founding stones of security which needs to be understood and deployed across the enterprise.

The use of digital signatures has seen tremendous growth in recent years and with the onset of new technologies, in particular Web-services, promises to be the dominant area in security. Corporate espionage is on the rise, and security can not be overlooked.
Ensure your system vulnerabilities are checked - Cross Site Scripting seems to be the worst offender in modern systems. Make sure your internet-facing applications are hosted on supported and patched platforms. Approach it with an outside-in, basic-first strategy for your IT department instead of focussing on obtuse things like bit-encryption levels first, ensure you can prioritize defenses against the most probably threat vectors first.

Software Architectures need to be evaluated.

What constitutes an architecture?

“You employ stone, wood and concrete, and with these materials you build houses and palaces. That is construction. Ingenuity is at work.

But suddenly you touch my heart, you do me good, I am happy and I say ‘This is beautiful’. That is Architecture.”

- Le Corbusier, 1923

- Quoted in Architecture: From Prehistory to Post-modernism

Well, then what is software architecture?

There is no universal agreed upon formal definition of software architecture, however, the Software Engineering Institute (SEI) has defined it as follows:

“The software architecture of a system is the structure of structures of the system, which comprise software components, the externally visible properties of those components and the relationships among them.” - SEI’s definition of Software Architecture.

- It is a vehicle for communication among stakeholders.

- It is the manifestation of the earliest design decisions.

- It is a reusable, transferable abstraction.

Software elements – modules, components etc. Externally visible properties – does provide for internal flexibility. E.g. a contract is externally visible.

All designs involve tradeoffs. Architecture is the earliest life-cycle artifact that embodies significant design decisions: choices and tradeoffs.

Predict a system’s quality attributes by studying its architecture. We can analyze architecture for achievement of quality attributes – it determines risk not a “grade”.

Bottom line: an evaluation should result in architectural “Risks Themes”. See SEI’s web-site for details.

Embrace Change Don't Just Criticize The Architecture

I love change when it fuels positive growth and innovation. It excites me because I can see ideas that once lived only in my head transform into real products. I enjoy crafting solutions that people can hold, use, and benefit from. That moment, when vision becomes reality, is deeply rewarding to me. The creativity offered by software architecture, user interface design, and backend engineering hits the sweet spot for me.

But in my journey, I’ve often come across software architectures that didn’t start with a vision. They simply “evolved out of need.” Sometimes teams “end up” with architectures that just happened to them, patch by patch, release by release. Other times, projects begin with a neatly sketched design, but even those evolve under the weight of deadlines, tradeoffs, and shifting requirements: Technical Debt.

And that’s not necessarily a failure. Every architecture tells a story of its context.

When speaking to decision makers, I often use analogies. Think about it like buildings. You might look at a structure and think, “Wow, that looks ugly.” But to the contractor, it may be a lucrative project. To the people living inside, the aesthetics may not matter at all. They may say if it ain't broken don't fix it. Architecture, whether in concrete or code, is always a balance between risks, non-risks, tradeoffs, and sensitivity points. I learned that at Carnegie Mellon's Software Engineering Institute. These aren’t just abstract concepts; they are the risk themes that give us the clarity to make informed design decisions.

As a seasoned IT Architect, here's what I believe: no architecture is inherently good or bad. It only becomes meaningful when evaluated through the lens of context and risk. Once risks are surfaced, explained, and understood, the decision shifts from being “good versus bad” to “fit for purpose.”

ATAM

The ATAM process is a short, facilitated interaction between the stakeholders to conduct the activities outlined in the blackboard, leading to the identification of risks, sensitivities, and tradeoffs:

• risks can be the focus of mitigation activities, e.g. further design, further analysis, prototyping

• sensitivities and tradeoffs can be explicitly documented

Architecture reviews are not repeatable without a process. ATAM gives a defined process to achieve a repeatable architecture evaluation process.

The federally funded Software Engineering Institute Carnegie Mellon has pioneered this method for evaluation of software architectures.

Non-repudiation–not a non-issue

To understand electronic non-repudiation, we must understand traditional non-repudiation from a legal perspective. The basis for a legal repudiation of a manual signature can pass only if the signature is a forgery, or an authentic signature was obtained via unconscionable conduct by a party to a transaction, fraud instigated by a third party, and undue influence exerted by a third party (McCullagh & Caelli, 2000).

From a technical perspective non-repudiation (NR) is basically proof that a certain principal sent or received the message in question. Every message exchange can be tied to a principal with a guarantee. An NR token is generated and verified that is sent by the principal – this way the principal cannot deny sending that message. In the same way, an NR token for a message received by the principal is created – this way the receipt of the message cannot be denied either.

The technical meaning of non-repudiation shifts the onus of proof from the recipient to the alleged signatory or entirely denies the signatory the right to repudiate a digital signature (McCullagh & Caelli, 2000). The use of a trusted system can solve the authentication, authorization and consequently non-repudiation issues by leveraging digital signatures.

Web-services. With more and more e-commerce being conducted on the Web and business-to-business transactions occurring, the importance of non-repudiation and digital signatures has gained a lot of importance. In the future, digital signatures will be commonly used in this area for providing non-repudiation services to the enterprise.

Authorization with ACL

Authorization is the process by which valuable resources are protected and only limited access provided to principals who are authenticated. Principals are entities that request access to resources. Principals can be people or other servers. It is important to note that authorization can take place only when authentication of the principal has occurred previously. This makes sense because principals who are unable to prove their identity should not be given permission to access sensitive information.

Authorization in the Enterprise

In the enterprise environment access control comes in many flavors including discretionary, role-based, mandatory, and firewall types of access control. Discretionary authorization is the process by which two principal are given mutually exclusive access to the same resource. For example, principal A can be give read-only access to resource C while principal B can be given full access to the same resource. Usually such access control mechanisms are hierarchical in nature.

Access Control List

Discretionary access-control mechanisms typically maintain a list of principals and their associated permissions in an access control list (ACL). ACLs can be stored separately that can be accessed during the authentication or authorization process. Principals can also be parts of groups and have group access permissions applied. Role-base access control is applied when a usage role has to be applied across several principals. If there are multiple system users then a user group is created and a common ACL applied. Once the ACL is applied to the group, all principals that belong to the group automatically inherit the permissions too. It is still possible in most cases to override, overload or perform other polymorphic behaviors to user-permissions applied to principals. Applying access controls to security groups and principals works well in most cases.

Classifications. Classification levels may be used to specify authorization levels, in this scheme the resource, principal and groups are all supplied with a pre-defined authorization level, the level of comparative authorization defines the actual access roles. For example, if resource C is tagged as classified, resource D as unclassified, principal A as classified and principal B as unclassified then principal A can access both C & D while principal B can access only D. Such parallel hierarchies can determine the access logic with ease. In general, if a principal’s classification level is higher than that of the resource then the principal is given access to the resource.

Firewalls. Inter-network communications is often protected by a firewall in the enterprise. A firewall is a mechanism by which access to particular transport control protocol/Internet protocol (TCP/IP) ports on some network of computers is restricted based on the location of the incoming connection request. Firewalls are often a gateway that connects two or more networks. Rules can be applied to firewalls that can block certain ports, protocols and Internet protocol (IP) addresses from access the network. Proxy-servers are sometimes installed inside corporate networks that typically bypass the firewall.

Trust domains. Domains can be defined and be used to protect sensitive resources. This is accomplished by grouping all servers and processes that have the same access control policy into a domain. This trust-domain can interact at the micro level with a level of trust defined by the ACL. IP address with specific ports and communications can also be included in the domain as well. Security policy domains are also sometimes called realms.

Java technology. Java employs stringent security standards in the Java Virtual Machine (JVM), however when security domains are pre-defined, code can be executed over uniform resource locators (URL) within the trust-domains. Also multiple domains can be defined and trust at a certain level is defined, this way code executing in one domain can trust, and make useful calls to code running in another domain. The domain is thus called trusted domain. Sub-domains can be created and each sub-domain can have one or more parent domains. The partitioning of domains by creating sub-domains provides the ability to assign more restrictive permissions at the sub-domain level – but not higher access levels. Domains can also be federated; the federation of domains allows permissions to be assigned to domains and other sub-domains.

Auditing. Authorization requests can be logged by the servers, gateways and firewall. Audit logs can help isolate sequence of events of particular threads of events. Investigation of such type can be done in order to uncover any suspected authorization attempts into protected resources. A lot of information can be logged into the security log files. Typical information that is logged during a security audit is audit type events, timestamp of the event, identity of the principal requesting access, identification of the target source being requested, permission being requested on the target source, location from which the target source is being requested and any protocol-specific information respectively (Jaworski, Perrone & Chaganti, 2001). Due to the sensitive nature of security logs accessing security logs should be restricted to authorized principals.