Fazlul's Information and Communication Technology Blog

Data Science, Big Data & Microsoft Machine Learning

2018-11-06T23:27:00.001-08:00

Preface

Data is seemingly controlling us and everything around us in many ways. Big data is more prevalent in our daily life more than what we think. Business Intelligence has become integral part for effective decision making. Data Science, Artificial Intelligence (AI) and Big Data Analysis has been playing pivotal role in the digital transformation processes. Current industry trend is to assess data behaviour and build knowledge libraries.

Industires are dealing with billions of micro devices that are connected to the control hub and emitting time series data continuously. These devices need to be connected to the central data pipeline and thus predict or prescribe device's health, availability and possible failure points. To connect billions of devices into corporate data pipeline, we need to have Big Data repositories. This is where IoT concept kicks in. Dr. Williams, Director of MIT described a world, where "things" (devices or sensors) are connected and able to share data. Data coming from these devices and sensors provide business insights that were previously out of reach. The invaluable insights enabled by harnessing and analyzing the data from these connected devices are what the Internet of Things (IoT) is all about.

DCS, APCs, RTUs, SCADA, PI Systems, DVR and MOM fall under the category of OT while Infrastructure, Product Lifecycle, ERP and Logistics fall under IT umbrella. If we look into the following diagram, we can imagine why we need a full stack platform to take the idustry towards Stream Analytics.

Figure 1: IT/OT 5 Layer Architecture

Microsoft established Azure based IoT Cloud model, which has been regarded as one of the key products for Digital Transformation. Azure IoT platform can connect to your connected devices and gain insights. It can then turn those insights into action with powerful applications built on the industry-leading platform for IoT development. From manufacturing to transportation to retail, start fueling new revenue and business opportunities with IoT solutions designed for industry needs.

Recently, Microsoft released Microsoft Machine Learning platform to provide Big Data analysis capabilities. Microsoft Machine Learning Server incorporates three renowned advanced & statistical analytic platforms which are: R, Python & Anaconda. All of the platforms have their pros and cons, there are some distinct advantages and disadvantages with each of them. My main focus of this document will be on R or previously known as Revolution R.

Julia White, CVP Microsoft Azure mentioned that Microsoft will invest $5 billion in the Internet of Things (IoT) over the next four years. So, it's worth looking at Microsoft Machine Learning platform as you could integrate with other amazing products from Microsoft stack like SQL Server for Database, Reporting & Analysis Services, Power BI for Business Intelligence & Analysis Services, BizTalk for Data Integrations Services.

Data Science

Data Science is a multidisciplinary field that involves scientific methods, algorithms and platforms to extract valuable insight of data from different perspective. This process monitors the behavioural pattern of data over the period of time, traing identified data models and thus predict or prescribe the futuristic flow of the data. In order to solve complex problems, it emphasises on data inference, data exploration & insights, algorithm/model development. Overall, Data Science encompasses Data wrangling, Specificity, Machine Learning by developing and traning specific Data Models.

Big Data Analysis

Big Data Analysis is done thorugh some specialized tools, platforms and processes to extract the insights from large volume of data. It can corelate differnt types of similar streams of data which was impossible through ordinary tools and platforms. Hadoop, SAS, R and Phython are some powerful advanced big data analytics platforms that goes beyond the normalized databases or databases that follows only data represented by rows and columns. It can derive insights from structed or unstructured databases or large data files.

According to the book "Making the World Work Better" by Kevin Maney, Steve Hamm, Jeffrey O'Brien, 1200 exabytes (which is equal to 1200 billion gigabytes) of digital information were created in 2010 which has been rising exponentially since then. Because of the high demand of disk spaces, hard drive manufacturing companies like segate, western digital, sandisk, samsung are focusing on producing cheaper hard drives with maximum storage capacities.

Machine Learning

Machine learning is basically broad class of methods that emphasises on Predictions & Pattern Discovery. According to SAS, Machine learning is a method of data analysis that automates analytical model building. It is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention. The processes involved in machine learning are similar to that of data mining and predictive modeling. Both require searching through data to look for patterns and adjusting program actions accordingly.

These are some widely used algorithms being used for Machine Learning:

- Data Co-Relations

- Decision Trees

- K-Means Clustering

- Neural Networks

- Reinforcement Learning/Deep Learning

Figure 2: Machine Learning Process

Microsoft Machine Learning Platform

Microsoft released a Machine Learning Platform previously known as "R" or Revolution Analytics platform. This platform is being delived by keeping in mind the need of Data Analysts. It can be used for both Offline/Online and Structured/Unstructured Data. Microsoft's machine learning platform has been elaborated briefly in the following section:

1. Microsoft Machine Learning Server (Standalone)

Microsoft Machine Learning Server is an in-data advanced analytics offering that enables us to work with flexible tooling across the existing open source and enterprise IT investments.

This offering works within Microsoft's end-to-end data science solution to empower organizations to solve business problems and seize new opportunities by enabling them to analyze data where it lives, innovate using artificial intelligence, and build mission-critical applications faster. It is a flexible enterprise platform for analyzing data at scale, building intelligent apps, and discovering valuable insights across your business with full support for Python and R.

Figure 4: Microsoft Machine Learning Server & R Client Communication

Feature category	Description
R-enabled	R packages for solutions written in R, with an open-source distribution of R and run-time infrastructure for script execution. Everything you had in R Server and more.
Python-enabled	Python modules for solutions written in Python, with an open-source distribution of Python and run-time infrastructure for script execution.
Pre-trained models	For visual analysis and text sentiment analysis, ready to score data you provide.
Deploy and consume	Operationalize your server and deploy solutions as a web service.
Remote execution	Start remote sessions on a Machine Learning Server on your network from your client workstation.
Scale out on premises	Clustered topologies for Spark on Hadoop, and Windows or Linux using the operationalization capability built into Machine Learning Server.

Table 1: Key features of Microsoft Machine Learning Server

2. Microsoft Machine Learning Services (In-Database)

SQL Server 2017 Machine Learning Services is an add-on to a database engine instance, used for executing R and Python code on SQL Server. Code runs in an extensibility framework, isolated from core engine processes, but fully available to relational data as stored procedures, as T-SQL script containing R or Python statements, or as R or Python code containing T-SQL.

SQL Server 2017 Machine Learning Services is the next generation of R support, with updated versions of base R, RevoScaleR, MicrosoftML, and other libraries introduced in 2016.

Figure 3: SQL Server Machine Learning (In-Database) Services Execution

Microsoft Machine Learning Services (In-Database) Components:

Component	Description
SQL Server Launchpad service	A service that manages communications between the external R and Python runtimes and the database engine instance.
R packages	RevoScaleR is the primary library for scalable R. Functions in this library are among the most widely used. Data transformations and manipulation, statistical summarization, visualization, and many forms of modeling and analyses are found in these libraries. Additionally, functions in these libraries automatically distribute workloads across available cores for parallel processing, with the ability to work on chunks of data that are coordinated and managed by the calculation engine. MicrosoftML (R) adds machine learning algorithms to create custom models for text analysis, image analysis, and sentiment analysis. sqlRUtils provides helper functions for putting R scripts into a T-SQL stored procedure, registering a stored procedure with a database, and running the stored procedure from an R development environment. olapR is for building or executing an MDX query in R script.
Microsoft R Open (MRO)	MRO is Microsoft's open-source distribution of R. The package and interpreter are included. Always use the version of MRO installed by Setup.
R tools	R console windows and command prompts are standard tools in an R distribution.
R Samples and scripts	Open-source R and RevoScaleR packages include built-in data sets so that you can create and run script using pre-installed data.
Python packages	Revoscalepy is the primary library for scalable Python with functions for data manipulation, transformation, visualization, and analysis. Microsoftml (Python) adds machine learning algorithms to create custom models for text analysis, image analysis, and sentiment analysis.
Python tools	The built-in Python command line tool is useful for ad hoc testing and tasks.
Anaconda	Anaconda is an open-source distribution of Python and essential packages.
Python samples and scripts	As with R, Python includes built-in data sets and scripts.
Pre-trained models in R and Python	Pre-trained models are created for specific use cases and maintained by the data science engineering team at Microsoft. You can use the pre-trained models as-is to score positive-negative sentiment in text, or detect features in images, using new data inputs that you provide. The models run in Machine Learning Services, but cannot be installed through SQL Server Setup.

3. Microsoft R Client

Microsoft R Client is a free, community-supported, data science tool for high performance analytics. R Client is built on top of Microsoft R Open so we can use any open-source R package to build your analytics. Additionally, R Client includes the powerful RevoScaleR technology and its proprietary functions to benefit from parallelization and remote computing.

R Client allows us to work with production data locally using the full set of RevoScaleR functions, but there are some constraints. Data must fit in local memory, and processing is limited to two threads for RevoScaleR functions. To work with larger data sets or offload heavy processing, you can access a remote production instance of Machine Learning Server from the command line or push the compute context to the remote server. Learn more about its compatibility.

Machine Learning Server and Microsoft R Client offer virtually identical R packages, but each one targets different scenarios. R Client is intended for data scientists who create solutions that run locally. Machine Learning Server is commercial software that runs on a range of platforms, at much greater scale, with infrastructure for handling major workloads, on client-server topologies that support remote access over authenticated connections.

- From R Client, we can shift data-centric RevoScaleR operations to a remote Machine Learning Server by creating a remote compute context. Remote compute context is supported for SQL Server Machine Learning Services or a Spark cluster. Typically, we shift the compute context to bring computations to where the data resides, thus avoiding data transfer over the network.

- From R Client, we can run arbitrary R code on a remote production instance of Machine Learning Server. This is a general-purpose capability: from a command line, you can switch between local and remote sessions interactively, useful for testing, administration, or to use the additional processing power of a production server.

Figure 5: Microsoft Machine Learning Server & Client Sessions

4. Microsoft R Open

Like, Microsoft R Client, Microsoft R Open is a free product. Microsoft R Open is the enhanced distribution of R from Microsoft Corporation.

Microsoft R Open includes:

- The open source R language, the most widely used statistics software in the world

- The installation of many packages include all base and recommended R packages plus a set of specialized packages released by Microsoft Corporation to further enhance your Microsoft R Open experience

- Support for Windows and Linux-based platforms

Plus these key enhancements:

- Multi-threaded math libraries that brings multi-threaded computations to R.

- A high-performance default CRAN repository that provide a consistent and static set of packages to all Microsoft R Open users.

- The checkpoint package that make it easy to share R code and replicate results using specific R package versions.

Samples

NYC Taxi demo data

Scope: Drawing a Histogram plot on Tipping Ratio for NYC Taxi demo

Code:

Ø library(RODBC)

Ø sqlConnString <- odbcDriverConnect('driver={SQL Server};server=DHA00730-ESCD02; database=NYCTaxi_Sample; trusted_connection=true')

Ø sqlData <- sqlQuery(sqlConnString, "SELECT tipped, fare_amount FROM nyctaxi_sample;")

Ø rxHistogram(~tipped, data=sqlData, col='lightblue', title = 'Tip Histogram', xlab ='Tipped or not', ylab ='Counts');

Result:

Scope: Trained data model for NYC Taxi demo

Code:

Ø sqlConnString <- odbcDriverConnect('driver={SQL Server};server=DHA00730-ESCD02;database=NYCTaxi_Sample; trusted_connection=true')

Ø sqlData <- sqlQuery(sqlConnString, "select tipped, fare_amount, passenger_count,trip_time_in_secs,trip_distance, pickup_datetime, dropoff_datetime, dbo.fnCalculateDistance(pickup_latitude, pickup_longitude, dropoff_latitude, dropoff_longitude) as direct_distance from nyctaxi_sample tablesample (70 percent) repeatable (98052);")

Ø logitObj <- rxLogit(tipped ~ passenger_count + trip_distance + trip_time_in_secs + direct_distance, data = sqlData)

Ø summary(logitObj)

Result:

References

1. Microsoft Machine Learning Server, https://docs.microsoft.com/en-us/machine-learning-server/what-is-machine-learning-server

2. Microsoft R Client, https://docs.microsoft.com/en-us/machine-learning-server/r-client/what-is-microsoft-r-client

3. Microsoft R Open, https://mran.microsoft.com/rro

4. NYC Taxi demo data for SQL Server Python and R tutorials, https://docs.microsoft.com/en-us/sql/advanced-analytics/tutorials/demo-data-nyctaxi-in-sql?view=sql-server-2017

SAP RFC Integration through BizTalk

2017-12-12T23:08:00.000-08:00

It has been a long while since I published some technical write-ups into my blog. Today, I will try to elaborate on Integration mechanism with SAP though BizTalk middleware. As you are aware, SAP is a self-contained and modular system. So, communicating with SAP is a complex process. You have to have basic understanding of SAP Modules and Data Query/Retrieval process. But, communicating to SAP through RFC function calls has made it even easier. You can communicate with SAP RFCs in three different ways:

1. Access SAP RFCs using WCF-SAP Adapter and SAP system credentials

2. Access SAF RFCs through exposed RFCs as ABAP web services

3. Access SAF RFCs through SAP Process Orchestrator, which is basically the SAP integration middleware

Today, I will concentrate on No. 2, which is to communicate with SAP through RFC Web Services via ABAP web services. To do that, you need to expose SAP RFCs as consumable web services. Please follow THIS to expose RFCs as ABAP web services and follow THIS link to expose RFCs as SOAP web services through SAP Process Orchestrator. But, in either ways you should be able to communicate with SAP with system credentials.

I will be using Visual Studio to demonstrate the BizTalk Orchestration development.

Step 1: Create a BizTalk Project

Step 2: Add Generated Items to the Project

Step 3: Consume WCF Service

Step 4: Consume Meta Data from a set of (*.WSDL) files

Step 5: Add the WSD Path and File Name

Step 6. Click Import and it will import the Service Description to the BizTalk Project and it will automatically create necessary Schema Definition for SAP Web Service, SAP Request Response Port Type and a Blank Orchestration file.

Step 7: Open up the Orchestration file and add the Receive, Transform, Send, Receive, Transform and Send shapes in to the blank orchestration in the following order

Step 8: Add a Schema (i.e. InputSchema.xsd) to the Solution for the Parameter Input file and add a Child Record (i.e. InputParameters) and add necessary Child elements to it.

Step 9: Add another Schema (i.e. OutputSchema.xsd) to the Solution for the Query output file and add a Child Record (i.e. Locations) followed by another child record (i.e. Location) for Looping through multiple item. Then add necessary Child elements to it.

Step 10: Open the Solution Window in Orchestration View and add the necessary Messages items and select appropriate Message Types (i.e. InputSchema /OutPutSchema /SAPRequest /SAPResponse)

- InputMessage:

- OutputMessage:

- RequestMessage:

-ResponseMessage:

Step 11: Add necessary Ports/Port Types as follows:

- Input parameter port (One Way Receive Port Type/Receiving Messages) - ParamReceivePort

- SAP Request/Response Port (Request Response Port Type/Sending a Request and Receiving a Response) – SAPRequestResponse, N.B. Sap Request response port type will be created while importing the WSDL. So, you need to select existing port type in this case.

- Output file port (One Way Send Port Type/Sending Messages) – WriteToFilePort

Step 12: Assign corresponding Message Types/Operations to ReceiveInput, SendRequest, ReceiveRequest and SendOutput. You will see that the Ports and Message Shapes are now connected.

Step 13: Update Message Constructed for 2 Transform Shapes:

Step 14: Click on first Transform Shape (i.e InputToRequest) of Construct Message shape (TransformFromXMLInputToSAPRequest) and select Imput and Output Messages as follows. It is being used to transform Input Message to a SAP Request Message. It will create a Map (i.e BizTalk.SAIF.InputToSAP) and open it in design mode.

Step 15: Define the first Mapping from Input Parameter to SAP Request. To do that Add Input File Schema (i.e. InputSchema) in the Source Schema pan and add SAP Request Schema (i.e. Z_I_MIA_LOC_LIST) in the destination schema pan and link associated elements on both sides

Step 16: Click on first Transform Shape (i.e ResponseToOutput) of Construct Message shape (TransformFromSAPResponseToOuput) and select Imput and Output Messages as follows. It is being used to transform SAP Response Message to Output Message. It will create a Map (i.e BizTalk.SAIF.SAPToFileMap) and open it in design mode.

Step 17: Define the second mapping (i.e BizTalk.SAIF.SAPToFileMap) from SAP Response to Output. To do that add SAP Response Schema (i.e. Z_I_MIA_LOC_LISTResponse) in the source schema pan and add Output File Schema (i.e. OutputSchema) in the Destination Schema pan and link associated elements on both sides:

Step 18: Now, the orchestration is complete and ready to deploy. Now, we need to add application name/BizTalk Server name in the Deployment tab.

Step 19: Add a key file to make the assembly strongly typed.

Step 20: Now deploy the solution to BizTalk server. We need to open up BizTalk administration console and configure the BizTalk application that we have just deployed.

Step 21: Create a Receive port to receive input parameter file.

- Right click on Receive ports and select New->One-Way Receive Port

- Give it a name (ParamReceivePort)

- Click on Receive Locations->New

- Enter Receive Location Name (i.e. ParamReceivePortLocation), Type as FILE and Receive Pipeline as XML Receive.

- Click on Configure and select Receive Folder location and click "OK" and "OK" again to go back to Receive Port configuration

- Click on Inbound Maps and select the Input Schema

Step 22: Create a Send port to send request to SAP and get response back from SAP as well. Select "Static Solicit-Response Send Port" in this case.

- Give it a name (i.e. SAPRequestResponsePort)

- Select Type as WCF-Custom, Send Pipeline as XML Transmit and Receive Pipeline as XML Receive

- Click "Configure" button to configure WCF-Custom

- Enter URI and SOAP action header from the imported WSDL file for the SAP RFC. You will get it almost towards the end of the file

- Click on Binding tab and select "Basic" as Client Credential type

- Increase incoming message size (if needed) by clicking on BasicHttpBindingElemtent->maxReceiveMessageSize

- Enter SAP User ID and Password in the Credentials tab and click "OK"

- Then click on Filters to subscribe to Receive Port. This is necessary to read the input parameter file from Receive Port. Enter BTS.ReceivePortName == ParamReceivePort (i.e. the Receive Port Name)

- Click "OK" to save the SAP Request/Response port configuration

Step 23: Create a Send port to receive input parameter file.

- Right click on Send ports and select New-> Static One-Way Send Port

- Give it a name (WriteToFilePort), Type as FILE and Send Pipeline as XML Transmit.

- Click on Configure and select Send Folder location and click "OK" and "OK" again to go back to Send Port configuration

- Click on Inbound Maps and select the Input Schema

- Click on Filters and add BTS.SPName == SAPRequestResponsePort. This step is necessary to add a subscription to SAPRequestResponsePort to get the SAP Response into the Output port

Step 24: Now we are all set. We just have to configure the BizTalk application and select Host Instances and assign the Physical ports to Orchestration ports. So, right click on the application and select "Configure" and select corresponding items from dropdowns.

Step 25: You can generate the "Input Schema" file from Visual Studio by selecting "Generate Instance".

Step 26. Update the InputSchema file with required parameters and drop it into the Receive Location and you will see a new file with GUID has been created in to the Send Location.

VOILA !!!

Cloud Computing Technology Assessment

2016-10-23T06:00:00.001-07:00

Introduction

Definition of Cloud Computing

Cloud Computing Service Models

Cloud Computing Deployment Models

Why Cloud Computing?

Benefits and Drawbacks of Cloud Computing

Survey Results on Cloud Computing

Microsoft Office 365 and Windows Azure Cloud Model

Preface

By the word "Cloud" in technology term, we mean an online service through which we can get access to the contents as well as functionality any time and from anywhere. It is a real shift in computing mindset. The main shift is to move from "Software as a Product" concept to "Software as a Service" concept. Cloud computing is gaining popularity for its ease of access and worry free maintenance. But, there are lots of concerns about the Legal aspects of putting sensitive and confidential information into the grip of third party cloud service providers or online hackers. Industries are also thinking of TCO (Total Cost of Ownership), the Speed and Availability of online service applications etc.

Introduction

Cloud computing has gained immense popularity in the market. Ease of creating a full blown operational environment has made it very attractive. Getting access to the services from anywhere has added extra value to the new trend of cloud computing. There are still some legal challenges to put confidential information into the cloud.

Let us see what Cloud Computing is and why it is different than traditional computing. According to the definition of NIST, Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction[i]. A Cloud Service provider will provide the virtual infrastructure or service into the cloud to quickly build and deploy applications without Purchasing any Hardware/Software, Maintaining or Upgrading the Infrastructure, overall getting access to the information anytime, anywhere and from any device.

Our aim is to find out details about Cloud Computing, Advantages and Disadvantages of Cloud Technology, Benefits of Cloud computing over the Traditional Computing, Cloud Computing Architecture and Cloud Service Providers.

Definition of Cloud Computing

The NIST definition is one of the clearest and most comprehensive definitions of cloud computing and is widely referenced in US government documents and projects. This definition describes cloud computing as having five essential characteristics, three service models, and four deployment models. The essential characteristics are:

On-Demand Self-Service: Computing resources can be acquired and used at any time without the need for human interaction with cloud service providers. Computing resources include processing power, storage, virtual machines etc.

Broad Network Access: The previously mentioned resources can be accessed over a network using heterogeneous devices such as laptops or mobiles phones.

Resource Pooling: Cloud service providers pool their resources that are then shared by multiple users. This is referred to as multi-tenancy where for example a physical server may host several virtual machines belonging to different users.

Rapid Elasticity: A user can quickly acquire more resources from the cloud by scaling out. They can scale back in by releasing those resources once they are no longer required.

Measured Service: Resource usage is metered using appropriate metrics such monitoring storage usage, CPU hours, bandwidth usage etc.

Cloud Computing Service Models

Cloud Services are being offered based on different service models. There are various Could Service providers in the market of which Microsoft, VMWare, Amazon and Google are the ones to be mentioned. Cloud Service Offering and Pricing Model also varies from provider to provider.

Software as a Service (SaaS):

The capability for the customer to use the provider's applications running on the provider's cloud infrastructure. The applications are broadly accessible from an Internet browser. Although the customer does not manage any of the underlying infrastructure, the customer manages content and possibly some limited configuration settings.

This is where users simply make use of a web-browser to access software that others have developed and offer as a service over the web. At the SaaS level, users do not have control or access to the underlying infrastructure being used to host the software. Salesforce's Customer Relationship Management software and Google Docs are popular examples that use the SaaS model of cloud computing.

Platform as a Service (PaaS):

The capability for the customer to deploy application tiers, such as a database or a web service, to a container managed by the provider. The customer does not manage operating systems or infrastructure in this model, but has control over their application, data, and possibly some configuration settings.

This is where applications are developed using a set of programming languages and tools that are supported by the PaaS provider. PaaS provides users with a high level of abstraction that allows them to focus on developing their applications and not worry about the underlying infrastructure. Just like the SaaS model, users do not have control or access to the underlying infrastructure being used to host their applications at the PaaS level.

Infrastructure as a Service (IaaS):

The capability to provision storage, archive, network, and server resources in a shared environment. Typically, server resources are deployed as virtual machines (VMs). The management of the server operating systems and other resources within this environment is performed by the customer, but the management of the underlying hardware and hypervisor is performed by the provider.

This is where users acquire computing resources such as processing power, memory and storage from an IaaS provider and use the resources to deploy and run their applications. In contrast to the PaaS model, the IaaS model is a low level of abstraction that allows users to access the underlying infrastructure through the use of virtual machines. IaaS gives users more flexibility than PaaS as it allows the user to deploy any software stack on top of the operating system. However, flexibility comes with a cost and users are responsible for updating and patching the operating system at the IaaS level.

Cloud Computing Deployment Models

Private Cloud: A Private Cloud being hosted within or outside and operated by the organization itself or by a Third Party provider but the use is exclusively by the same Organization.

Public Cloud: A Public Cloud is a common cloud for everyone. The Cloud service can be purchased based on Type of Service Models And / Or Number of Users and Type of Service being accessed. Public Clouds require significant investment and are usually owned by large corporations such as Microsoft, Google or Amazon.

Community Cloud: A Communality Cloud is generally shared by several organisations that shares the common goal.

Hybrid Cloud: Hybrid Cloud is the combination of Different Cloud Deployment models. In most cases, industries keep their confidential or corporate data in Private Cloud or On Premise environment and publish only the required information into the cloud.

Why Cloud Computing?

Vast majority of industries have taken initiatives to move partially/completely to Cloud or to look into the Cloud services from Cost, Maintenance, Availability and Performance point of view. Another big concern is, the Application providers are concentrating on Cloud as their platform of Service. Let's take a closer look at the Cloud predictions for the future[ii]:

More application availability on the cloud

Software product development companies are concentrating their effort on building Cloud based applications. Most of them are providing both solutions (Cloud and On-Premise) but most advanced features are being implemented in the Cloud Solution. Microsoft will release the new and advanced features into Office 265 before releasing it for SharePoint On-Premise server.

With most new software being built for cloud from the outset, it is predicted that by 2016 over a quarter of all applications (around 48 million) will be available on the cloud (Global Technology Outlook: Cloud 2014: A More Disruptive Phase).

Increased growth in the market for cloud

Based on their forecast for 2011-2017, Gartner expects adoption to hit $250 billion by 2017. In the same forecast, Gartner also suggested that the worldwide software as a service (SaaS) market would grow at an astounding yearly growth rate of 20.2 percent! This means it will be growing from $18.2 billion in 2012 to $45.6 billion in 2017.

More hybrid cloud adoption

By using Hybrid cloud structure industries can go into cloud into phases or separate out the sensitive or confidential/legal information and keep them into the corporate network whereas publish the rest of it into the Cloud. Gartner proposes that 50 percent of enterprises will have hybrid clouds by 2017. As we see more and more companies adopt cloud, we see CIOs crafting well-thought-out strategies that include cloud. However, pure cloud implementations are the exception and not the rule.

Increased development for the cloud

Software Product development companies started building products that is Cloud Based or compatible with certain cloud services. Cloud based developments are increasing significantly. According to Evans Data Corporation, there are more than 18 million software developers worldwide yet less than 25 percent are developing for the cloud today.

More innovation because of cloud

To make Cloud computing popular ad interactive, more innovations are coming into the Cloud Computing. As the Cloud based products are generally device independent, lots of innovations are coming to make cloud experience more interactive and attractive.

Benefits and Drawbacks of Cloud Computing

The Benefits of Cloud Computing

There are some significant benefits of Cloud Computing of which the following ones are the ones to be mentioned:

Flexibility

Cloud computing is highly flexible regarding the expansion of the operating environment. A cloud-based service can instantly meet the demand because of the vast capacity of the service's remote servers. Based on Salesforces' Information Week survey[iii], 65% of respondents said "the ability to quickly meet business demands" was an important reason to move to cloud computing.

High Availability

In Cloud environment, it is possible to add the servers into the availability group and service provider increases/decreases resources when required to make the Cloud environment Highly Available. Once the application is deployed, provisioning, load-balancing, health monitoring, are being handled by the cloud service provider. Based on SLAs the uptime of the applications are higher than On Premise deployments and may range up to 99.95% uptime in some cases.

Disaster recovery

In case of any disaster, the recovery is much faster in Cloud Computing. Cloud Service providers have online backup systems in place and being synced continuously.

Automatic software updates

Cloud Service Providers apply software updates automatically without disrupting the service. Customers don't have to think of applying software patches or software upgrades any more in Cloud Computing SaaS and PaaS models.

Auto Scalable and Upgradable

Cloud services are auto scalable. In case of more resources, it can be adjusted on the fly without affecting the application. But, the options may vary based on the service agreement for the subscribed services.

Application Development vs. Infrastructure Maintenance

Cloud solutions increases productivity by concentrating on the Actual development and delivery rather than spending much time on maintaining the System in terms of Hardware, Software, and Networks etc.

Convenient health monitoring and alerts

Cloud Service providers provide convenient health monitoring through Cloud dashboards or Alerts. It is very effective and free tool that is being used to identify any probable issue that will hamper the environment. Health and Monitoring is mostly part of the package for IaaS.

Cost Savings

Cloud Computing Cost Model is usage based and charged "pay-as-you-go". The cost model is a shift from Capital Expenses to Operating Cost and reduces total IT expenses. Maintenance and Support Costs are already included into the package. So, there is a big savings from the ma hours to maintain the infrastructure.

Increased collaboration

Cloud Computing increases collaboration between employees from different cities or countries. It keeps them all connected.

Any Device, Any Where

Cloud computing models support working from anywhere as long as employees have internet access. Most Cloud services are available from Any Device (i.e. iPad, iPhone, Windows Phone, Windows Tablets, and Android Phones/Tablets etc.). It increases productivity and supports "Bring Your Own Device" concept.

The Drawbacks of Cloud Computing

Security and Privacy

The information security and privacy are the major concerns over the cloud from user perspective. By leveraging a remote cloud based infrastructure, sensitive, confidential and legal information/documents are being hosted inside Cloud Service Provider's environment. In Multi-Tenant environment, the Data Center or Hosting Infrastructure is being shared by multiple companies. From the legal point of view some information/files cannot be kept outside of certain cities or countries. Keeping these aspects in mind, a careful decision has to be made.

Companies rely on Cloud Service providers to protect their data from un-authorized access or usages. Any un-authorized access could be a big threat for the company.

Performance

Depending on the type of application we deploy into Cloud Services, Performance factor may play a big role depending on which Data Centers we are connecting to and how much is the bandwidth is available from the Cloud Infrastructure.

Limited Control

Limited control is being provided for the deployment and administration of Cloud Services based for PaaS and SaaS deployment models. These deployment models lack of features compared to the Local Platforms or Applications.

Increased Vulnerability

Cloud based solutions are exposed to Internet and hence becomes more vulnerable target for malicious users and hackers. If security is being compromised, corporate data may be exposed to whole internet. This why strict security model need to be adopted and multi-factor authentication may be prescribed.

Technical Difficulties and Downtime

Cloud Computing is fully dependent on Stable Internet connections. If there is a problem in the internet, it may affect its whole purpose. Besides Internet, they could be some downtime or failure from Cloud service providers which may affect the entire company while accessing relevant information on time.

Survey Results on Cloud Computing

There have been lots of surveys done on Cloud Computing. I am going to share some of the survey results:

North Bridge Venture Partners (Mid 2014):

North Bridge Venture Partners, Gigaom Research and supported by more than 70 other collaborating organizations conducted the largest survey on Cloud Computing till Mid-Year of 2014[iv]. The survey analyzed the drivers and inhibitors behind cloud adoption, comparing the Cloud with Traditional technology with a sample of 1,358 respondents.

Cloud Adoption is Strategic

49% of respondents in this survey are using cloud to develop new products or applications

45% of businesses are already or planning to go to the Cloud

SaaS Adoption

SaaS adoption increased from 11% (2011) to 74% (2014)

Sales and Marketing adopted 51 %, whereas customer service and analytics both adopted at 43%.

IaaS and PaaS Adoption

56% of businesses are using Infrastructure-as-a-Service (IaaS)

41% of businesses are using PaaS technologies to prototype and develop new applications

2/3rd of respondents believe that their data will come to reside in some form of cloud over the next 2 years

Microsoft IT Pro Cloud Survey[v]

Microsoft conducted a survey on cloud computing. 1,979 people took part in this survey and the results are discussed below:

Cloud Computing Involvement:

Most important benefits of the cloud (Benefit from 1 to 5 with 1 being the most important):

Greatest Barrier in Cloud Computing:

Types of Cloud Computing:

Impact of cloud computing (Next 5 Years):

Knowledge of Cloud:

With cloud and hybrid services expected to reach US$108 billion by 2017[vi]. Microsoft also released their Cloud Service sales update brief of which is given below:

Microsoft Office 365:

77% of Fortune 500 companies have purchased it in the past 12 months

With a US$2.5 billion run rate, it's Microsoft's fastest growing business

Microsoft Office 365 posted triple-digit growth for 21 consecutive months

Microsoft partners have accounted for 3/4 of the sales to Microsoft Office 365 customers

Microsoft Azure:

57% of Fortune 500 companies are using Microsoft Azure

Welcomes 1,000 new customers per day Currently 1.2 million businesses and organizations use Microsoft Azure Active Directory

Microsoft Azure gains two times the compute and storage capacity every 6-9 months

Microsoft Office 365 and Windows Azure Cloud Model[vii]

Microsoft's Cloud Platform named Windows Azure is well known in the Cloud Computing world. Amazon, Google, IBM are also proving cloud services at different levels of services. Microsoft Cloud provides added benefit by bringing Microsoft Office experience into the Cloud. Office 365 is Microsoft's Cloud product for SaaS (Software as a Service) and Windows Azure is the host platform for PaaS (Platform as a Service) and IaaS (Infrastructure as a Service) service offerings. Let us discuss details of Microsoft Cloud Service models.

SaaS (Software as a Service) – Office 365/CRM Online

Software as a Service (SaaS), meaning delivering software over the Internet -- provides the ability to simplify deployment and reduce acquisition costs. Microsoft Online Services are subscription-based, on-demand applications and hosted services, providing your organization with a consistent experience across multiple devices. Microsoft's SaaS model includes the following products:

Microsoft Exchange Online delivers the email system in the Cloud. Businesses can migrate or host their email system into the Cloud without requiring any on premise infrastructure for Exchange.

Microsoft SharePoint Online and Power BI provides the SharePoint Online for Collaboration and Content Management whereas Power BI for full blown Business Intelligence platform for the enterprise.

Microsoft Office Professional Plus is available in select Microsoft Office 365 service plans and provides Microsoft Office through web browsers and from any device without practically having Microsoft Office installed into the machine.

Microsoft Lync Online provides the online communication platform for the business. Employees can communicate through Voice, Video Conferencing, and Texting/Chatting through Online presence

Microsoft Dynamics CRM Online delivers rich sales automation, service management, and marketing automation functionality through the browser and within everyday productivity applications of the business

Office 365 includes Exchange, SharePoint, and Office Professional plus as a part of their combined SaaS cloud offering whereas Dynamics CRM and Power BI are offered as a part of different service model.

PaaS (Platform as a Service) - Windows Azure

PaaS stands for Platform as a Service. In this mode, it's possible to deploy and run customized applications/services with limited control. The basic infrastructure, network topology, load balancers are all configured automatically. Windows Azure PaaS provides with data services, including a relational database, reporting and data synchronization. Both Windows Azure and SQL Azure are the key components of the Azure Cloud Platform. It is easy to scale up or scale down SQL server according to the application need. One of the key benefits of PaaS is that we don't have to be concerned about the Operating System, Platform Features, Upgrade or Maintenance.

IaaS (Infrastructure as a Service) - Windows Azure

In IaaS model, full infrastructure can be outsourced to the Cloud including Virtual Servers, Storage, Networking, Load Balancers ad so on. The full infrastructure or network can be accessed through secured internet. In SaaS model, it's the subscribed business's responsibility to maintain the Infrastructure including Patches/Updates or regular monitoring. Azure IaaS is pretty much same as office infrastructure except it is in the Cloud. IaaS subscription model depends on computing power that Business is using (No. of Cores, memory, HDD, network, bandwidth etc.). But, the easy part is to provisioning a new server. New server can be built based on existing server templates (LINUX, UNIX, Windows, SharePoint etc.) within 5-10 minutes. IaaS provides granular control to the infrastructure and additional resources can be added, upgraded or removed from Virtual Machine management console.

Hybrid Cloud Architecture

Hybrid Cloud is the combination of different Cloud Deployment models. A hybrid cloud is the combination of at least one public and one private cloud, with the workload distributed across the two. In most cases, industries keep their corporate data in the Private Cloud or On Premise environment and publish only the required information into the Public cloud.

In Hybrid Cloud environment, we have to segregate the applications and services between On Premise/Private Cloud and Public Cloud. In some cases, we may have to extend the existing applications from Premise/Private Cloud to Public Cloud. While extending applications and services into another cloud is determining how to ensure that access to these services is consistent for the users. So, we have to get the answers of the following questions before approaching Hybrid Cloud platform:

· Will services deployed in the public cloud run directly on the Internet?

· How will you manage identity for internet-facing services?

· Do those services require access to systems and data in the private cloud?

If we need the services to connect to the Private Cloud or On Premise databases/applications, careful consideration needs to be made about how to connect these services while protecting corporate/sensitive data and networks. A public cloud like Azure offers many different ways to connect these services. It also offers Windows Azure Service Bus for secured and scalable way to communicate between services through internet-standard protocols. But, the traditional option is also available to communicate through secured VPN tunnels. For large scale deployments, Microsoft is investing in high-speed interconnect options like MPLS connections to Azure with some cell carriers. Hybrid cloud can be separated by web service layers or consumable service API's from Public Cloud. Hybrid applications can be designed by adopting IT policy, SLAs, Security, and Data Sovereignty. SLA will dictate application will be placed in Private or Public Cloud. Based on Data Security policy and legal implications data store can be inside corporate network whereas the application could be in the Public.

[i] The National Institute of Standards and Technology (NIST) is an agency of the U.S. Department of Commerce.

[ii] http://thoughtsoncloud.com/2014/05/future-cloud-computing-5-predictions/

[iii] http://www.salesforce.com/uk/socialsuccess/cloud-computing/why-move-to-cloud-10-benefits-cloud-computing.jsp

[iv] http://cloudtweaks.com/2014/06/2014-future-cloud-computing-survey-results/

[v] http://download.microsoft.com/documents/uk/technet/cloud-power/ITPro_survey_v5.pdf

[vi] https://mspartner.microsoft.com/en/hk/Pages/Solutions/cloud-is-our-core.aspx

[vii] http://www.microsoft.com/en-us/download/details.aspx?id=18467

http://blogs.technet.com/b/in_the_cloud/archive/2013/12/16/success-with-hybrid-cloud-best-practices-for-building-a-hybrid-cloud.aspx

SharePoint Server 2013 Infrastructure Standard

2014-12-19T01:58:00.001-08:00

SharePoint Farm Standards

Windows Server 2012 R2 with Update should be used as the Operating System for SharePoint hosting servers
SharePoint 2013 SP1 (with KB2880552) should be used as the SharePoint platform for Company1 Energy unless it is well justified and approved
SQL Server 2012 SP1 should be used as the back-end database and it has to be aligned with Company1's Database Standards
If possible, SharePoint Servers should be created based on a pre-defined template followed by the following steps:

- Install any Windows/SharePoint Service Packs/Microsoft Updates

- Install any other SharePoint Updates

- Install add-ons or third party components, if applicable

- Install and Log/Backup/Monitoring Tool

SharePoint Production and UA Farms should be installed and configured following three tier architecture of SharePoint farm:

Web Tier should host WFEs (Web Front Ends) to serve web pages and Host query components
Application Tier should host Application Servers for Central Administration, Service Applications/Services, Search/Crawl Components
Database Tier should host Configuration Database, Content Databases, BI/Reporting Database, and Search/Crawl/Property Databases that is part of the SharePoint Farm. The databases are separated based on the services on the server. Generally Search, Content/Config and BI/Service Application Databases are separated

Services on server

SharePoint Capacity Planning

Small farm deployment

A small farm deployment consists of a single database server or cluster and one or two SharePoint Server 2013–based computers. The major architecture characteristics include limited redundancy and failover, and a minimal set of SharePoint Server 2013 capabilities enabled.
A small farm is useful to serve only limited deployments, with a minimal set of service applications enabled, a relatively small user base, a relatively low usage load (a few requests per minute up to very few requests per second), and a relatively small volume of data (10 or more gigabytes).

Medium farm deployment

This architecture introduces the breakdown of the topology into three tiers: dedicated Web servers, dedicated application servers, and one or more database servers or clusters. Separating the front end server tier from the application server tier allows greater flexibility in service isolation and helps balancing the load across the system.
This is the most common architecture, and includes a wide spectrum of service topologies and farm sizes. A medium farm deployment is useful to serve environments that have the following:
Several service applications distributed across multiple servers. A typical set of features might include the Office Web Apps Service, User Profile Service, Managed Metadata Service, and Excel Calculation Service.
A user base of tens of thousands of users and a load of 10 to 50 requests per second.
A data store of one or two terabytes.

SharePoint Development servers may be used as the single server for WFE, App or DB servers
Developer should use their own Virtual Machines for SharePoint Development
In case of implementation of Business Intelligence features in SharePoint, a separate SharePoint Application server should be used
If On Base or any other application needs to be integrated with Enterprise Search, a separate SharePoint Application server should be used to separate Search from other service applications
If possible, SharePoint servers should be created from the base template rather than installing SharePoint on brand new Windows 2012 Servers
For unattended service accounts, secure store target application IDs should be used
Predefined SharePoint Managed Accounts should be used for SharePoint Application pools or any other services
Scaling out a farm with server groups: In SharePoint Server 2013, the number of services and corresponding databases is greater than in previous releases. The recommendation for scaling out a farm is to group services or databases that have similar performance characteristics onto dedicated servers and then scale out the servers as a group. For example, group all client-related services onto one or two servers and then add servers to this group as needed to satisfy user demand for these services. In some cases, we might need to create a dedicated server group for a single service, such as Excel Services or Search. This model groups service applications and related components (for example, databases) into several different logical groupings that can be used as a starting point. In large environments, the specific groups that evolve for a farm depend on the specific demands for each service.

Database Server Standards

"Security Hardening" should be implemented in SharePoint Farm.

Blocking the standard SQL Server ports: Custom port should be used for SQL server instead of using default port which is 1433
Use Named instances "SHAREPOINT" of Database Server instead of Default instance
SQL Server database instances should be configured to listen on a nonstandard port

through a Non-Standard port.

Grant SVCSPAdmin or SVCSPAdminDEV or SVCSPAdminUA (depending on the environment) dbcreator and securityadmin SQL roles
Grant SVCSPFarm or SVCSPFarmDEV or SVCSPFarmUA (depending on the environment) dbcreator and securityadmin SQL roles
SQL Server client aliases should be used instead of Physical Server names: A SQL Server client alias "SHAREPOINTSQL" on all SharePoint Servers should be created and used to communicate with Database Server
Set max degree of parallelism (MAXDOP) to 1 for instances of SQL Server that host SharePoint databases to make sure that a single SQL Server process serves each request.

Important: Setting the max degree of parallelism to any other number can cause a less optimal query plan to be used that will decrease SharePoint Server 2013 performance.

Ref: http://technet.microsoft.com/en-us/library/hh292622(v=office.15).aspx

Naming Standards

We should follow the standard naming convention across all environments and environment specific naming should not be used
The content databases could be named starting with SP_ (followed by name format and name) and there should not be any blank spaces inside/outside of the names (e.g. Central Admin content database could be SP_Content_CentralAdmin, Intranet content database could be SP_Content_Intranet and Applications content database could be SP_Content_Applications)

Role	Name Format	Example(s)
Configuration Database	sp_config	sp_config
Content Database	sp_content_<function>	sp_content_admin sp_content_portal sp_content_mysites
Single Database Service Application	sp_service_<service application name>	sp_service_mms sp_service_state
Multiple Database Service Application	sp_service_<service application name>_<database name>	sp_service_ups_profile

While creating Sites/List/Libraries/etc. the blank spaces should be avoided and description may have blank spaces after its being created. (e.g. Architecture, TruckTransportation, CanwestPropane etc.). In this way there won't be any "%20" in the URLs

Hardware Standards

To deploy Microsoft® SharePoint® Server at Company1, we must meet the following requirements:

Hardware: 64-bit

SharePoint Server 2013 and SharePoint Foundation 2013 are 64-bit applications and can only run on a 64-bit edition of Windows Server® 2012 or 2008/2008 R2 operating systems. We must have hardware that supports the use of a 64-bit operating system and 64-bit SQL Server.

Hardware — Web servers, application servers, and single server installations

The requirements in the following table apply both to installations on a single server with a built-in database and to servers running SharePoint Server 2013 in a multiple server farm installation.

Component	Minimum requirement
Processor	64-bit, 4 cores (For Standard SharePoint) 64-bit, 4 quad-cores (For PowerPivot Deployment)
Memory	16 GB (For Standard SharePoint) 24 GB (For Single Development Server running all available services) 64 GB (For PowerPivot Deployment)
Hard disk	OS: 120 GB (C: Drive) Logs: 20 GB (D: Drive) We must have sufficient space for the base installation and sufficient space for diagnostics such as logging, debugging, creating memory dumps, and so on. For production use, we also need additional free disk space for day-to-day operations. Recommended free space needs be of 5 times of memory. But, maintain at least twice as much free space as we have memory for production environments.

Hardware — Database servers

The requirements in the following table apply to the database servers in production environments with multiple servers in the farm. But, this recommendation has to be aligned with Company1's Database Server policy and standards.

Component	Minimum requirement
Processor	64-bit, 4 – 8 cores
Memory	16 GB for DB Server Memory
Hard disk	System Drive(C:) – 40 GB Binaries (D:) – 40 GB Data (E:) – 100 GB (More for Production but based on Contents) Log (F:) – 40 GB Temp (T:) - As Required Logs: Company1 standard database server Hard disk space is dependent on the size of the SharePoint content. The sample calculation is being given below.

Database Size:

The size of SharePoint content database depends on some factors. Based on requirements and capability the size of databases or servers has to be determined.

Database size: (D.V.S) + (10 KB . (L + (V.D)))

D: number of documents
V: number of non-current version
S: Average size of document
L: list items

Ex: (200,000 . 2.250) + ((10 KB . (600,000 + (200,000 .2))) = 110,000,000 KB or 105 GB

Service Accounts

SharePoint needs some pre-defined service accounts to install and configure SharePoint. SharePoint will elevate the following accounts as necessary during installation. So, for Non Production (Dev/Test) and Production separate service accounts should be created. Like:

For Production Environment "SVCSPAdmin" will be " SVCSPAdminPRD"

For UA Environment "SVCSPAdmin" will be " SVCSPAdminUA"

For DEV Environment "SVCSPAdmin" will be " SVCSPAdminDEV"

Account name	Role	Domain rights	Local SharePoint Server rights needed	SQL rights needed
SVCSPAdmin	Used to install SharePoint binaries.	Domain User	Local administrator on all SharePoint boxes	dbcreator and securityadmin SQL roles
SVCSPFarm	Farm account. Used for Windows Timer Service, Central Admin and User Profile service	Domain User	Local Admin during UPS provisioning, log on locally right	None
SVCSPWebapp	App pool id for content web apps	Domain User	None	None
SVCSPService	Service app pool id	Domain User	None	None, unless using Office Web Apps. Them must give access to content databases manually
SVCSPSearch	Search process id	Domain User	None	None
SVCSPContent	Account used to crawl content	Domain User	None	None
SVCSPUserprofile	Account used by the User Profile services to access Active Directory	Must have Replicating Change permissions to AD. Must be given in BOTH ADUC and ADSIEDIT. If domain is Windows 2003 or early, must also be a member of the "Pre-Windows 2000" built-in group.	None	None
SVCSPPowerPivot	PowerPivot Service Account	Domain User	None	None
SVCSPPerformancePoint	PerformancePoint Service Account	Domain User	None	None
SVCSPExcel	Excel Services Service Account	Domain User	None	None
SVCSPVisio	Visio Service Account	Domain User	None	None
SVCSPSuperuser	Cache account	Domain User	Web application Policy Full Control Web application super account setting	None
SVCSPSuperreader	Cache account	Domain User	Web application Policy Full read Web application super reader account setting	None

DNS Names

It is recommended to use the DNS names (http://spsandbox) instead of physical server address and port number (http://MYSERVER1:80) for each web application in a SharePoint Environment. The DNS names are being used to assign the host headers for the web applications.

Environment	Web Application	DNS Name
Company1 Team Sites
Sandbox	Intranet Team Sites	http://team.sb.Company1.com
Development	Intranet Team Sites	http://team.dev.Company1.com
UA	Intranet Team Sites	http://team.ua.Company1.com
Production Support	Intranet Team Sites	http://team.ps.Company1.com
Production	Intranet Team Sites	http://team
Company1 Applications
Sandbox	Application Site	http://apps.sb.Company1.com
Development	Application Site	http://apps.dev.Company1.com
UA	Application Site	http://apps.ua.Company1.com
Production Support	Application Site	http://apps.ps.Company1.com
Production	Application Site	http://apps
Company1 Intranet (Inmotion)
Sandbox	Intranet Publishing Site	http://intranet.sb.Company1.com
Development	Intranet Publishing Site	http://intranet.dev.Company1.com
UA	Intranet Publishing Site	http://intranet.ua.Company1.com
Production Support	Intranet Publishing Site	http://intranet.ps.Company1.com
Production	Intranet Publishing Site	http://intranet
Company1 Public Facing Corporate Site
Sandbox	Internet Publishing Site	http://www.sb.Company1.com
Development	Internet Publishing Site	http://www.dev.Company1.com
UA	Internet Publishing Site	http://www.ua.Company1.com
Production Support	Internet Publishing Site	http://www.ps.Company1.com
Production	Internet Publishing Site	http://www.Company1.com
Company1 Public Facing Extranet Site
Sandbox	Extranet Team Sites	http://extranet.sb.Company1.com
Development	Extranet Team Sites	http://extranet.dev.Company1.com
UA	Extranet Team Sites	http://extranet.ua.Company1.com
Production Support	Extranet Team Sites	http://extranet.ps.Company1.com
Production	Extranet Team Sites	http://extranet.Company1.com

SharePoint BI - How to fix SSRS Issues

2014-08-21T12:41:00.001-07:00

SQL Server Reporting services can be used and configured in Integrated Mode and Native mode as well. There are some noticeable differences between both modes.

SSRS in SharePoint Integrated Mode:

• Use SharePoint document management and collaboration features. A SharePoint site provides a unified portal for accessing and managing all documents in one place.

• Use SharePoint permissions and authentication providers to control access to reports, models, and other items.

• Manage reports, models, data sources, schedules, and report history in custom application pages on a SharePoint site. We can set properties, define schedules and subscriptions, and create and manage report history on a SharePoint site the same way you create and manage them from other tools in SQL Server.

• Publish or upload reports, report models, resources, and shared data source files to a SharePoint library, including Report Center in Office SharePoint Server.

• Use Report Designer, Model Designer, and Report Builder to create reports and data sources to be published directly to a SharePoint library. We can also use the Upload action on a SharePoint site to add any report definitions and report models to a SharePoint library.

• Any report that we can run in a native mode report server can run on a report server that is configured for SharePoint integrated mode.

• Subscribe to and deliver reports to a SharePoint library using a new SharePoint delivery extension. We can also deliver reports through e-mail or to a shared folder. The report server delivery extensions are used to deliver reports. You can create data-driven subscriptions for large-scale report distribution using subscriber data queried at run time.

• Using a Report Viewer Web Part we can add to SharePoint pages to view a report inside a SharePoint Page. The Web Part includes page navigation, search, print, and export features.

SSRS in Native Mode:

• In native mode, a report server is a stand-alone application server that provides all viewing, management, processing, and delivery of reports and report models. This is the default mode for report server instances.

• Reports are being hosted in a separate server in Native Mode and report URLs are added in to the SharePoint Pages or Web Parts.

• When we open a report by clicking the link in SharePoint page, SharePoint redirected to the Report Server URL.

• When we open a report through SharePoint Web Part, it renders the report by from the link/Report URL provided in SharePoint page

In case of some recent patches or for some other reason if we cannot run the reports or are getting errors regarding rendering, encryption of some unknown errors, we may have the check the Reporting services DLLs (Native Mode) or Reporting Services Applications (Integrated Mode) and see if they are working properly. As the SSRS native error fixes are related to SQL Server, lots of the issues can be identified and fixed through SSRS Report server.

But, in case of SSRS in SharePoint integrated mode we have to look into SSRS Service and SSRS service Application. We have to check if the SQL Reporting Service is in running state in Manage Services on Server in SharePoint Administrator.

Then, we have to check the SSRS Service Application settings (like: execution account, settings etc.). In case of total failure we may have to delete the SSRS service Application and re-create it without deleting the SSRS Database. The steps are as follows:

Step 1: Export the Encryption Key to a file

Go to SSRS Service Application and select Key Management

Then select Backup Encryption Key

Enter a password

And Save it in a secured location.

Step 2: Delete SSRS Service Application

Delete existing SSRS Service Application without deleting the SSRS Database (i.e. ReportService_SharePoint). Go to the Application Properties and remember the Application Pool and the SSRS Database Name (Take a snapshot before deleting)

Then delete the SSRS service application

Step 2: Create SSRS Service Application

Create a new SSRS Service Application pointing to the existing SSRS Database (i.e. ReportService_SharePoint).

N.B. Use the previously created application pool with same Service Account.

Step 3: Import Encryption Key

1. Import previously exported Encryption Key to SharePoint SSRS service

Step 4: Reset Execution Account and IISRESET

2. Reset the Execution Account for SSRS

3. Do an IISRESET

Boom !!!! done…..everything will be back online…..

SharePoint Hosting Solution - Which One to Choose? - Part 1

2014-02-05T14:02:00.001-08:00

SharePoint is a technology platform quite similar to other technology platforms that provide certain features and functions for certain specific business needs. So, it basically goes down to Business Requirements, Size of an Enterprise, Enterprise Governance and what we need to achieve by implementing this technology platform. There are lots of slick technologies in the market and may not be suited for the specific types of business or work that we are considering. Second thing that comes into play is the vision and goal for that specific technology and calculating the ROI as a whole. Third step is to road mapping that technology which needs to be aligned with the future state of the enterprise. Before jumping into recommended hosting solution, I wanted to elaborate some key aspects of implementing SharePoint in the Enterprise.

Vision Statement:

Now, let us talk about the vision statement for SharePoint which could be similar to this or beyond this:

“The Portal provides an environment where, in one organized, intuitive and searchable location employees will find the systems, tools, information and collaboration areas that they need. From any internet-connected computer, employees will be able to realize greater efficiency accessing important business systems, people information, company forms, news, benefits summaries, calendars and other company information. The portal will provide tools for teams to collaborate and share documents, timelines, status reports and other communication, thus providing additional benefit to our customers, and our industry partners in a secure and efficient way.”

Enterprise Goals:

What follows are the key enterprise goals as they relate to what SharePoint will and won’t be used to support.

SharePoint Will Be:

§ Utilized by Business Users to develop and implement business solutions that use technology without IT’s direct involvement.

§ Utilized by IT to develop and implement more advanced technical solutions through a unified application delivery platform.

§ The primary repository for document sharing, collaboration, and communication.

§ The primary location for simple workflow/business process automation.

§ The employee Portal and communication center for internal corporate communications.

§ The primary project issue tracking system with the support of Project Server.

SharePoint Will Not Be:

§ The repository for any XYZ documents and processes.

§ The platform used for customer relationship management.

§ The learning management system used by our organization.

§ Utilized as an asset management system.

§ The ticket tracking system for our operational issues.

Primary Goals:

1. Improve Communication: Implement a Portal in SharePoint in order to improve communication between the organizations departments, and employees.

2. Foster Collaboration: Implement a structure within SharePoint to enable the sharing and collaboration of files and documents through SharePoint’s built-in functionality in order to move away from reliance on file shares.

3. Connect Employees: Improve the ability for employees to find each other and improve team collaboration using SharePoint’s built-in people search capabilities.

4. Improve Productivity: Provide greater efficiency to employees by simplifying access to important business systems.

5. Unify Communication: Implement all [Company] Departments by the end of _______ for a more unified experience and to further improve communication between departments and employees.

6. Optimize Collaboration: Improve the sharing and collaboration of files and documents through SharePoint’s built-in functionality in order to move away from reliance on file shares.

7. Accelerate Discovery: Provide the ability to search for and find relevant files and documents using SharePoint’s built-in search capabilities both in and outside of SharePoint.

8. Encourage Engagement: Enable employees to share more personal information through their profiles and the built in social features of SharePoint.

Some Research Findings:

A Forrester Research Inc. survey of IT decision makers shows 62 percent plan to deploy SharePoint 2013 on-premises, while only 8 percent will do so in a Microsoft datacenter, presumably via Office 365. Only 4 percent will run SharePoint in the datacenter of a Microsoft partner, while 26 percent will deploy SharePoint in a hybrid mode, both on-premises and online.

Another survey recently released by Metalogix Software Corp. had similar findings, showing 55 percent intend to continue running SharePoint entirely in-house and only 10 percent plan to run it purely in the cloud. The remaining 35 percent are planning hybrid SharePoint implementations.

SharePoint Hosting Solutions:

SharePoint Online/Office 365:

SharePoint Online delivers the powerful features of SharePoint without the associated overhead of managing the infrastructure on your own. Flexible management options ensure that you still retain the control you need to meet the compliance requirements of your organization. You can purchase SharePoint in the cloud as a standalone offering or as part of an Office 365 suite where you could also get access to Exchange, Lync, the Office clients and web apps. The Public Website in SharePoint Online – which is part of Office 365 – can be used to quickly and easily create a professional online presence for your business or organization. The website is ideal for small and midsize businesses because it contains simple design tools to create and maintain a website on the Internet.

Cross device availability

Easily access and interact with your SharePoint newsfeed, wherever you go using the SharePoint mobile apps available across various devices.

Easy to Manage

You can get setup in virtually no time! The powerful admin console allows your organization to easily manage capabilities, policies and security for all the content and features within SharePoint. Automated maintenance of the servers ensure that you are always up to date with the latest features with minimal downtime.

Enterprise grade reliability and standards

Safeguard your data by hosting it in geographically distributed datacenters with continuous data backup, premier disaster recovery capabilities and a team of experts monitoring the servers around the clock.

Public Website:

The Public Website in SharePoint Online is a hosted site that can be customized using just a web browser. Once you sign into Office 365, you can immediately add and edit pages, change the site theme, optimize the site for SEO, add social media apps, and much more. You can even use advanced site design features like Design Manager to create a custom design from scratch. When finished, you can make the website public and then add a custom domain name (for example, contosobistro.com). It’s a great way to quickly and easily create a website without having to know how to edit HTML, upload pages, or build website features. If you look at the Office 365 service, you’ll see Public Website listed as one of the available features, along with hosted email, file sharing, web and mobile apps, and more. The website is a SharePoint Online site collection and it’s available in Office 365 for Small Business, Office 365 for Midsize Business, and Office 365 for Enterprise.

The Public Website in Office 365 includes several new features that help you create a professional website for your business or organization.

Feature	Description
Design features	New website customization options, including a new Site tab, composed looks, social media, website blog, publishing features, Design Manager, and much more.
Management features	New site management features to manage permissions, storage, SEO, search reports, design galleries, make the website public, change its web address, and more.
App marketplace	Browse, install, and manage the use of apps from the SharePoint Marketplace.
Friendly URLs	Friendly URLs are now applied to the web pages on the Public Website. For example, the Contact Us page might look like www.website.com/contact-us/
Search Engine Optimization	Using new SEO features, you can improve the appearance and ranking of your public website in search engines like Bing and Google.
Domain names	Using the new domain features in Office 365, you can add, update, and remove custom domain names applied to the Public Website.

Plans for every business size and need

Office 365 Small Business Premium

Office 365 Midsize Business

Office 365 Enterprise

Big business power with easy IT

Equip your business with virtually anywhere access to the latest Office applications plus cloud-based IT services like enterprise-grade email and conferencing that are simple to administer.

Effective management, compliance and team collaboration

Give your people virtually anywhere access to the latest Office applications, cloud-based IT services like enterprise-grade email and team sites and powerful management and compliance tools.

Advanced tools and capabilities

Get the power your organization requires: virtually anywhere access to the latest Office applications, enterprise-grade cloud services like team sites and hosted voicemail and advanced IT capabilities.

What’s included in Office 365 Enterprise

FEATURE	DESCRIPTION
Office suite	Always have the latest versions of: Capture your ideas however you work best—using a keyboard, pen, or a touchscreen. Easily format information in Excel with tools that recognize your pattern and auto-complete data. Easily incorporate content from PDFs to create your own great-looking Word documents.
Office on more devices	Enjoy a consistent and familiar Office experience across PCs, Macs, Windows tablets, and most mobile devices.
Office on any PC	Stream full versions of Office programs to any Internet-connected PC running Windows 7 or Windows 8 with Office on Demand.
Advanced email	Use archiving and legal hold capabilities, plus unlimited storage, for compliance needs. And use data loss prevention (DLP) policies for additional compliance enforcement.
Document and email access control	Rights Management Services enables you to restrict access to documents and email to specific people and to prevent anyone else from viewing or editing them, even if they are sent outside the organization.
Voicemail integration (Unified Messaging)	Hosted voicemail support with auto-attendant capabilities. Voicemails are recorded to Exchange Online and users can access them from Outlook, Outlook Web App, or a compatible mobile phone.
Advanced business intelligence tools	Get the insight you need fast with advanced dashboards with full Excel Services. Perform big data analyses and data visualization mash-ups with the new Excel client with Power Pivot and Power View.
Email and calendars	Use business-class email through a rich and familiar Outlook experience you can access from your desktop or from a web browser using Outlook Web App. Get a 50 GB mailbox per user and send attachments up to 25 MB.
Web conferencing	Host online meetings with audio and video using one-click screen sharing and HD video conferencing.
Instant messaging and Skype connectivity	Connect with other Lync users via instant message, voice calls, and video calls, and let people know your availability with your online status. Share presence, IM, and audio calling with Skype users.
Enterprise social network	Yammer is a private social network that helps you and your company stay on top of it all. Get connected to the right people, share information across teams, and organize around projects so you can go further, faster.
File storage and sharing	SkyDrive Pro gives each user 25 GB of personal cloud storage that can be accessed from anywhere and that syncs with their PC for offline access. Easily share documents with others inside and outside the organization and control who can see and edit each file.
Team sites	Work together on projects with team sites that help keep related documents, notes, tasks, and conversations organized together.
Office Web Apps	Create and edit Word, OneNote, PowerPoint, and Excel documents from any modern browser.
Mobility	Sync email, calendar, and contacts; access SharePoint sites; view and edit Office documents with Office Web Apps using a browser on Windows Phone, iOS, and Android devices.
Office Mobile Apps	Stay productive and never miss a meeting, even on the go. Access, edit, and view Word, Excel, and PowerPoint documents on iPhones, Android phones, and Windows Phones. Use the OneNote, Lync Mobile, and SharePoint Newsfeed apps on most devices.
Administration	The admin portal provides IT detailed configuration options for your services, either from an online portal or through automated management with PowerShell commands.
Reliability	Get peace of mind knowing your services are available with a guaranteed 99.9% uptime, financially backed service level agreement (SLA).
Security	Cutting-edge security practices with five layers of security and proactive monitoring help keep customer data safe.
Privacy	Your data belongs to you. Microsoft does not scan emails or documents for advertising purposes.
Up to date	No need to pay for version upgrades; updates are included in your subscription. New features are rolled out to Office 365 customers in an IT-configurable experience.
Support	24/7 phone support for all IT issues. For less urgent issues, you can make service requests directly through the admin portal.

SharePoint Server

SharePoint Server Farm is a full blown SharePoint Infrastructure that can be setup On Premise, Cloud or even Hybrid model where external facing sites or extranet/internet sites can be hosted in the cloud with ADFS/FBA/AD authentication and intranet sites can be hosted On Premise. For much larger or complex website needs, such as Web Content Management, Business Connectivity Services to connect to internal applications and data, Web Analytics, More Customizations and Control or online commerce, we should consider a Full Blown SharePoint Farm which is hosted On-Premise or in the Cloud.

Some Key Features and Considerations while choosing the right SharePoint Hosting Solution:

Office 365 as a SaaS Solution:

Office365 SharePoint Online is a limited product which lacks the full set of features available in SharePoint. With Office365, Microsoft has trimmed down SharePoint Online vs. the traditional SharePoint product hosted On-Premise. To achieve the full functionality of SharePoint 2010, you need something more powerful than SharePoint Online. Only SharePoint private cloud hosting can deliver true parity with On-Premise hosting PLUS the cost savings of the Cloud.

Custom Web Parts: Installing these custom Web Parts - like Visual Web Parts, for example - may require direct administrative access to the underlying server components. But that means most types of custom-coded solutions won't be compatible with O365 since this degree of server control is restricted in SharePoint Online.

PowerPivot Support: Microsoft SQL Server PowerPivot enables tremendous reporting and analysis capabilities for large datasets stored within PowerPivot Excel Workbooks. So, installing and utilizing the SQL PowerPivot tools demands either On-Premise servers or private cloud hosting

SQL Server Reporting Services: SQL Server Reporting Services (SSRS) is an excellent data mining tool to gather business intelligence metrics and facilitate smart, efficient business decisions. With SSRS, enterprise data can be surfaced, analyzed and organized into intelligible reports and dashboards. Office365 doesn’t support running SSRS Reports on SharePoint.

Business Connectivity Services: SharePoint Business Connectivity Services (BCS) – aka SharePoint BDC – provides access to external data. That way critical business metrics can be imported from external sources directly into SharePoint applications. Unfortunately, though, the full Business Connectivity Services features are only available for On-Premise or private hosted SharePoint farms. BCS in SharePoint Online is limited to Windows Communication Foundation (WCF) applications; SharePoint Online is not capable of connecting with other types of external systems.

Web Analytics Customization: Analytics programs gather data about how visitors interact with and use your SharePoint-driven site to provide metrics such as time-on-page and length-of-visit. Yet this is another powerful SharePoint utility you won’t find in Office365. As a result, customizing Web Analytics reports demands using On-Premise infrastructure- OR, a hosted private cloud environment.

Site Collection Size: Office365 is capable of handling (at most) 100GB worth of site collection content. If your company has more than 100GB of SharePoint content databases, or if your current storage trends indicate you may quickly exceed that limit, it’s time to look for a hosting solution beyond Office365.

FAST Search Server: The FAST Search Server product enables enterprise-scale indexing and introduces new customizable functions that extend beyond the out-of-the-box search functions of SharePoint Server or Server Enterprise editions. A custom defined scope in SharePoint Search enables you to search the entire enterprise (File Share, Record Management Application like OnBase, SharePoint, Custom Applications or databases etc.) from same search box.

On-Premise or Cloud:

Now, let us talk about On Premise vs. Windows Azure or any other cloud hosted solution. While choosing the right solution, we should consider the business requirement, corporate policy and enterprise governance as a whole. There will always be some give and takes and we have to decide as an Enterprise which ones we want to take and which ones are we ready to consider. The table below shows the advantages and disadvantages of On-Premise vs. Cloud.

Advantages:

On-Premise:

· Control over all systems/data

· Corporate data is stored / Handled internally

· Dedicated IT staff for maintenance/support

· Initial investment is high but pays off over time

Cloud (Windows Azure/VMWare Cloud etc.):

· No software licensing costs

· No new infrastructure requirements eg. Servers

· Low cost for services

· Third parties are doing hard work

Disadvantages:

On-Premise:

· Lack of resources to build an infrastructure

· Software licensing costs are too expensive for a start-up

· Lack of resources, especially in IT

· Lack of technical know-how to maximize benefits of IT systems

Cloud (Windows Azure/VMWare Cloud etc.):

· Legal implications of putting sensitive data in Public cloud

· Third-parties are handling confidential data

· Redundancy – what happens if the solution provider fails?

· A subscription service could be expensive over time and as business grows

· Customization and integration with custom systems could be an issue

· Lack of full control over data and processes

Conclusion

Despite its limitations, SharePoint Online can be a smart solution for many types of clients. The simple, affordable pricing terms for Office365 may make this option more viable – especially for smaller companies with smaller number of (fewer than 100) users. Per-user monthly subscription pricing, with no need for upfront investment in hardware or software, is also a draw to the SharePoint Online platform.

But, for mid to large enterprises with Business Intelligence (Performance Point, Power View, Power Pivot etc.), Business Connectivity, Custom Solutions Development, Unique Policy rules, Powerful and fully-featured environment, Full blown SharePoint Server is the way to go.

I recommend implementing the Hybrid model if possible where the External facing SharePoint Farm is being hosted into the Cloud for availability from anywhere anytime and Intranet sites are being hosted internally and within the corporate network. A publishing model can be implemented to publish out required information from Internal SharePoint to the External SharePoint portals to facilitate external vendors or offsite employees. ADFS and FBA can be implemented in the External Farm environment where as NTLM/AD Authentication for Internal SharePoint Farm. Internal SharePoint Team sites can be published out to Citrix to provide employees get access to SharePoint from Home or Abroad.

For Development and Test environment, Cloud based and ADFS authenticated environment can be utilized to reduce the Dev/Test environment costing and getting the environment without any delay. A prior server template can be created to create the DEV/Test farms less than 10 minutes.

How SharePoint Can Benefit Your Enterprise

2013-11-01T09:45:00.001-07:00

Business Needs

· Information Worker

o Effective, consistent interface across business applications

o Immediate productivity

o Flexible, consistent user access

· IT Management

o Easy-to-deliver Web capabilities

o Simplified platform and content management

o Seamless access to business data

· Organization

o Strategic platform for Internet, extranet, and internet delivery

o Cost-effective content delivery

o Increased productivity

Internal Business Needs

· Connect and empower users

o Collaboration

o Building business communities and project groups

· Manage total cost of ownership

o Infrastructure

o Management

o Training

· Respond to business needs

o Change

o New opportunities

External Business Needs

· Maximize Internet opportunities

o Agile information distribution

o Building customer relationships

o Building brand

· Work seamlessly with partners

o Streamlining data accessibility

o Creating flexible shared systems

o Managing security

What's SharePoint?

· The business collaboration platform for the enterprise and the web

· Allows individuals in an organization to easily create and manage their own collaborative Web sites

· Simplifies how people find and share information across boundaries, and enabling better informed decisions

· Seamlessly integrates with Windows and MS Office

· Does not refer to a specific product or technology

· Using the word "Microsoft SharePoint" is like using the word "Microsoft Office"

· Refers to several aspects of Web-based collaborative solutions

SharePoint's Capabilities

· Enhance Document Management

· Promote better collaboration

· Improve reporting

· Automate business processes

· Integrate with existing tools

SharePoint as an Organizational Platform

· Individual groups can have a collaborative web site

· Access can be limited to the team and appropriate stakeholders

· Relevant information can be centrally stored and maintained

· Communications can be streamlined

· Relatively easy to use

o IT intervention is minimal

o Based on familiar tools and technologies: Web, Windows, Microsoft Office

· Technologies: Web, Windows, Microsoft Office

SharePoint Products

· SharePoint Online: A cloud-based service, hosted by Microsoft, for businesses of all sizes. Instead of installing and deploying SharePoint Server on premises, any business can now simply subscribe to an Office 365 plan, or the standalone SharePoint Online service, and their employees can create sites to share documents and information with colleagues, partners, and customers.

· SharePoint Foundation: The underlying technology for all SharePoint sites. SharePoint Foundation is available for free on premises deployment—in previous versions it was called Windows SharePoint Services. We can use SharePoint Foundation to quickly create many types of sites where we can collaborate on Web pages, documents, lists, calendars, and data.

· SharePoint Server: Organizations can deploy and manage SharePoint Server on premises. It includes all the features of SharePoint Foundation plus additional features and capabilities, such as Enterprise Content Management, business intelligence, enterprise search, personal sites, and Newsfeed.

· SharePoint Designer: A free program for designing, building, and customizing Web sites that runs on SharePoint Foundation and SharePoint Server. With SharePoint Designer, We can create data-rich Web pages, build powerful workflow-enabled solutions, and design the look and feel of the site. The sites we create can range from small project management team sites to dashboard-driven portal solutions for large enterprises.

· SharePoint Workspace: A desktop program that we can use to take SharePoint site content offline and collaborate on the content with others while we are disconnected from the network. While We and other team members are offline, We can make changes to SharePoint content that will eventually synchronize back to the SharePoint site. In SharePoint 2013, SharePoint Workspace was replaced with SkyDrive Pro folder sync.

· SkyDrive Pro folder sync: A desktop program that we can use to sync an offline version of a team site or SkyDrive Pro library to a folder on the computer.

SharePoint Composites

SharePoint Composites enable us to rapidly create solutions by assembling, connecting, and configuring the basic building blocks of functionality available in SharePoint Products 2013, and in many cases, Microsoft Office 2013. A SharePoint Composite combines data, documents, and business process in a useful, productive way.

In most cases, these solutions do not require code. Now there's nothing wrong with writing code. But here's a motto we might live by from now on: "Let's not write code until we have to write code."

In short, a SharePoint Composite is a "do-it-yourself" business solution. A SharePoint Composite could be as simple as a custom Web Page that uses a site list and several connected Web Parts to track mileage in a small business. Or, a SharePoint Composite could be as sophisticated as a corporate-wide absence reporting system connected to a Human Resource database by using Business Connectivity Services. A SharePoint Composite bears close resemblance to the often-used term, "mashup". A mashup is a quick Web application that incorporates data into a simple, visual, and interactive solution. However, the term "composite" emphasizes the breadth and depth of solutions we can build on the SharePoint Products 2013 platform.

Why SharePoint Composites?

To leverage existing investments: Just using out-of-the-box SharePoint Products 2013 saves us time, money, and resources. These are the tangible benefits for which we can readily see the cost savings to the business. But, creating a SharePoint composite can make the workforce more productive. Those are the intangible benefits that affect the bottom line and successfulness every day.

To help eliminate content and data silos: SharePoint Products 2013 is all about connection, collaboration, and helping to create a "shared brain" within the organization. The more we work in a cross-functional manner, the more We utilize the resources We already have, the more We realize the benefits of synergy and unification.

To unlock the value of Line-of-Business (LOB) data: With a SharePoint Composite based on Business Connectivity Services (BCS), We can work with data from other business systems as if the data lived within the SharePoint sites. Create, read, update, delete, and search the data using external content types and external lists. Surface enterprise data in Microsoft Excel 2013 and Microsoft Word 2013. Search across all the valuable business information. Work offline and synchronize the changes when we reconnect by using Microsoft Outlook 2013.

To rapidly create a business solution without dependence on technical staff: We don't need to be a programmer to assemble these building blocks. Without code, we can create highly customized sites and solutions, share data-driven Visio diagrams, and publish Microsoft Access 2013 Access apps. Use Microsoft SharePoint Designer 2013 and Microsoft InfoPath 2013 to design and build declarative solutions based on workflows and forms. We'll be surprised by how many of the business challenges can be resolved with a code-free solution.

Because we're a power user: Sometimes called a citizen developer, power users see colleagues struggling to do their work, think to themselves, "There has to be a better way", and see a problem that requires a solution they can create. Advancement in software, usability, and standards has made this a growing trend.

Because they are IT friendly: Both end users and IT want to tackle that proverbial project backlog and improve the overall business. But, there is a traditional struggle with the paradox of user responsiveness and empowerment versus IT control and governance. SharePoint Composites help resolve this paradox in an effective way. Solutions are visible and integrated into the organization's information infrastructure, and therefore are easier to deploy, manage, maintain, and ensure compliance.

Because they are developer friendly: Starting a new application from scratch is expensive, time-consuming, and risky. Enhancing a familiar, working solution is often a preferred path. From the perspective of a developer, a SharePoint Composite is like a working prototype that lays the foundation for an application, and that now needs to scale up to potentially thousands of users or millions of rows. Perhaps, We only need to add a custom Web Part. Or, we may need to revise the SharePoint Composite as a solution with custom, managed code. Either way, the upgrade path is clear and smooth.

7 Ways SharePoint Can Empower the Organization

#1 Easily Create a Collaborative Site

· Technical skill requirement is minimal

o Microsoft Windows

o Microsoft Office

o Familiar with web browsing

· Easily define relevant access

o Based on communication needs

· If deployed appropriately, IT does not have to deal with

o Updating content

o Defining account privileges

o Maintaining a document repository

#2 Efficiently Manage Information

· SharePoint provides various tools to effectively centralize

· and manage information

o Schedule

o Documents

o Change Request

o Risk/Issue Log

o Budget

· Document management features

o Information storage

o Check-in/check-out

o Version control

o Content approval

#3 Facilitate Team Collaboration

· Document Collaboration

o Document Workspaces can be used to jointly develop requirements document, reports, templates, etc.

· Tools

o Wikis to document lessons learned

o Discussion boards for offline communication

o Meeting Workspaces to support meetings

#4 Enhance Communication

· Right information for the right person at the right time

o Tasks

o Schedule

o Reports

o Dashboard

· Relevant information access

o Appropriate privileges can be defined based on informational needs

#5 Automate Business Processes

· Common project workflows

o Change Control

o Expense Reimbursement

o Vacation Request

· SharePoint workflows

o Three-State

o Custom workflows

#6 Generate Relevant Reports

· SharePoint can be used to generate relevant

o Interactive summary of a project

o Project tasks information

o Automated alerts

· Dashboards can be created using Web Parts

o Red, Amber, Green (RAG) Status

o Key Performance Indicators (KPI)

o Charts

#7 Integrate with Existing LOB Systems

· Integrate SharePoint with existing data sources

o SQL – based data

o Web Services

o XML

· Non Microsoft enterprise systems

o CRM

o Reporting Tools

References:

What is SharePoint by Microsoft

Document Link: http://office.microsoft.com/en-ca/sharepoint-server-help/what-is-sharepoint-HA010378184.aspx

What is SharePoint By Dux Raymond Sy.

Document Link: http://www.google.ca/url?sa=t&rct=j&q=&esrc=s&frm=1&source=web&cd=6&cad=rja&ved=0CFcQFjAF&url=http%3A%2F%2Fwww.bnl.gov%2Finterlab09%2FPresentations%2FWhat_is_SharePoint.pdf&ei=TW9yUv-kDceWiQKb4YHYDg&usg=AFQjCNEFAAfsB73bVpy124SZsyBLldzgyQ

SharePoint 2013 across multiple servers for a three-tier farm (Traditional Topology)

2013-09-26T14:56:00.001-07:00

Today, I will demonstrate how to create a three-tiered SharePoint farm using Traditional topology. If you want to go through Streamline topology, you will need couple more servers to diversify the services and service applications. In SharePoint 2013 Installation and Configuration of the Server types are pretty much same. But, you should create the App server first that hosts the Central Administration. Then you can go on creating the servers as many as you want by choosing "Connecting to Existing Farm" during the installation process. The DNS for the sites should point to the NLB or WFE as the sites are being hosted into the WFEs instead of APP servers. The App servers should be configured for SharePoint service applications like Search, BI. Managed Meta Data etc.

1.       Create Necessary SharePoint Accounts
2.       Install SQL Server 2012 with Named Instance for SharePoint: It is recommended to use named instances for SharePoint Database servers instead of Default instance to make it separate from other enterprise database servers.

3.       Setup custom TCP/IP port and Aliases for SQL Server: To make SharePoint scalable and easily maintainable, it is recommended to use the SQL Server Aliases. In that case, it is very easy to replace the existing server and replace with a new one or for any disaster management. Using a TCP/IP custom port, it makes SharePoint databases secured and reduces any external attack or injections. I will walk you through the processes step by step and by snapshots:

- Defining Custom TCP/IP Port for SQL Server

- Restart the SQL Server Service

- Check SQL Log to see if the SQL Server is listening to that custom TCP port

- Create an Inbound Firewall rule on the SQL Server to allow the inbound traffic on that custom TCP port for SQL Server (e.g. 65000)

- Create the SQL Server Aliases in DB, WFE and App Servers

4.       Grand DBCreator and SecurityAdmin to the SQL Server

5.       Install SharePoint 2013 Prerequisites followed by SharePoint 2013 Full Installation (App->WFE)
6.       Run SharePoint Configuration Wizard
7.       Turn on the following services on the following types of servers

- Web Front End Servers

Service Application	Services on Server	Server Recommendation	More information
	Claims to Windows Token Service	Web and application servers	This service is automatically configured to run on applicable servers.
	Distributed Cache	Web and application servers	By default this service is started on all web servers and application servers in a farm.
	Microsoft SharePoint Foundation Web Application	Web server	Ensure that this service is started on all web servers in a farm. Stop this service on application servers. This service provides web server functionality. It is started by default on web servers. Custom features scoped to web applications may not display in Central Administration as intended if this service is not started on the server that runs Central Administration and if the feature cannot be deployed globally.
	Microsoft SharePoint Foundation Workflow Timer Service	Web server	This service is automatically configured to run on all web servers in a farm.
	Microsoft SharePoint Foundation Incoming E-Mail	Web server or application server	Typically, this service runs on a web server. If you need to isolate this service, you can start it on an application server.
	Microsoft SharePoint Foundation Sandboxed Code Service	Web server or application server	Start this service on computers in the farm that run sandboxed code. This can include web servers and application servers. This service runs code that is deployed as part of a sandboxed solution in a remote, rights-restricted process and measures the server resources that are used during execution against a site collection-scoped, daily quota.
	Request Management	Web server or dedicated servers	In integrated mode, Request Management runs on all web servers in a farm. In dedicated mode servers in a separate Request Management farm are between the hardware load balancer and one or more SharePoint farms.

-Application Servers

Service Application	Services on Server	Server Recommendation	More information
Access Services 2010	Access Database Service 2010	Application Server	Allows continued maintenance of SharePoint 2010 Access service applications by using Access 2010 clients and Access 2013 clients. Does not allow users to create new applications.
Access Services	Access Services	Application Server	Allows creation of new Access service applications by using the Access 2013 client. View, edit, and interact with Access Services databases in a browser.
App Management Service	App Management Service	Application Server

Business Data Connectivity	Business Data Connectivity service	Application Server
Excel Calculation Services	Excel Calculation Services	Application Server
Machine Translation Service	Machine Translation Service	Application Server	The Machine Translation Service uses SharePoint OAuth for file management. The Machine Translation Service Application's app pool account needs permissions to the User Profile Application to function correctly. The Machine Translation Service connects to the Microsoft Translator online service. Microsoft Translator may collect some data to improve the quality of translations.
Managed Metadata Service	Managed Metadata Web Service	Application Server
Microsoft SharePoint Foundation Subscription Settings Service	Microsoft SharePoint Foundation Subscription Settings Service	Web server or application server	This service application is deployed only by using Windows PowerShell. In hosting environments, this service is typically started on one or more application servers. Start this service if you have deployed service applications in multi-tenant mode or if the farm includes sites that use site subscriptions. This service stores settings and configuration data for tenants in a multi-tenant environment. After it is started, web applications consume this service automatically.
PerformancePoint	PerformancePoint Service	Application server
PowerPoint Conversion	PowerPoint Conversion Service	Application Server	This service converts PowerPoint presentations to other formats. It typically runs on one or more application servers. It starts one or more worker processes to perform conversions. When actively converting, a worker process may use up a whole processor core. Memory usage depends on the size and content of files being converted. The number of worker processes that are used can be controlled through Windows PowerShell cmdlets. Several other configuration options are also available through Windows PowerShell cmdlets.
Search	Lotus Notes Connector	Application Server	Refer to TechNet documentation to learn how to configure Lotus Notes Connector which crawls data on a Lotus Domino server.
Search	Search Host Controller Service	Automatically configured to run on the appropriate application servers	This service manages the search topology components. The service is automatically started on all servers that run search topology components.
Search	Search Query and Site Settings Service	Automatically configured to run on the appropriate application servers	This service load balances queries within the search topology. It also detects farm-level changes to the search service and puts these in the Search Admin database. The service is automatically started on all servers that run the query processing component.
Search	SharePoint Server Search	Automatically configured to run on the appropriate application servers	This service crawls content for the search index. This service is automatically started on all servers that run search topology components. The service cannot be stopped or started from the Services on Server page.
Secure Store Service	Secure Store Service	Application server
Usage and Health Data Collection	NA	NA	This service application has no associated service on server.
User Profile	User Profile Service	Application server
User Profile	User Profile Synchronization Service	Application Server
Visio Graphics Service	Visio Graphics Service	Application server
Word Automation Service	Word Automation Services	Application server	Performs automated bulk document conversions. When actively converting, this service will fully use one CPU for each worker process (configured in Central Administration). If the service is started on multiple servers, a job will be shared across all the servers.
Work Management	Work Management Service	Application Server
	Central Administration	Application Server	This service runs the SharePoint Central Administration website.
	Claims to Windows Token Service	Web and application servers	This service is automatically configured to run on applicable servers.
	Distributed Cache	Web and application servers	By default this service is started on all web servers and application servers in a farm.
	Document Conversions Launcher Service	Application Server	Schedules and starts the document conversions on a server.
	Document Conversions Load Balancer Service	Application Server	Balances document conversion requests from across the server farm. Each web application can have only one load balancer registered with it at a time.

8. Create the appropriate DNS/Host Headers for SharePoint Sites and point them to the NLB or WFEs.

I have demonstrated the three tier architecture with traditional topology to install SharePoint 2013. But, if you have any issues, please feel free to contact me: mdfachowdhury@hotmail.com

How to install SharePoint 2013 with SSRS 2012 Integrated Mode

2013-05-31T11:21:00.000-07:00

SQL Server Reporting Services can be hosted and configured inside SharePoint using SharePoint Integrated mode installation. In this way the Report Server setting has to be setup and configured from SharePoint central administration. If we try to access the Report Server from outside, it will error out and which is only possible in case of SSRS Native mode installation in SQL 2012. The configuration of SSRS inside SharePoint administration is quite different for SQL 2012 Vs. SQL 2008 or SQL 2008 R2. And, as a part of standard best practices, we will be creating two different SQL Database instances or servers for SharePoint and SSRS. I will go through step by step installation and configuration procedure of SSRS 2012 in SharePoint 2013.

1. Operating System:

We can have this combination installed in two different Server Operating Systems (Windows Server 2012 or Windows Server 2008 R2 with SP1). But, as a part of this configuration, we will be using Windows Server 2008 R2 with SP1 as the Server Operating System.

2. Service Accounts:

SharePoint needs some pre-defined service accounts to install and configure SharePoint. SharePoint will elevate the following accounts as necessary during installation. So, for each environment we will be adding "Environment" for each service account. Like:

Account name	Role	Domain rights	Local SharePoint Server rights needed	SQL rights needed
SVCSPAdmin	Used to install SharePoint binaries.	Domain User	Local administrator on all SharePoint boxes	dbcreator and securityadmin SQL roles

SVCSPFarm	Farm account. Used for Windows Timer Service, Central Admin and User Profile service	Domain User	Local Admin during UPS provisioning, log on locally right	None

SVCSPWebapp	App pool id for content web apps	Domain User	None	None

SVCSPService	Service app pool id	Domain User	None	None, unless using Office Web Apps. Them must give access to content databases manually

SVCSPSearch	Search process id	Domain User	None	None

SVCSPContent	Account used to crawl content	Domain User	None	None

3. Install SQL Server & SharePoint Database:

As a Part of this farm configuration, we will be installing the SQL server and Database Engine services for SharePoint database only. This instance of SQL server will not be shared for any other types of SQL databases.

Step A:

Step B:

Step C:

4. Install SSRS and Database:

Now, we need to install SQL Reporting Services and Reporting Services Add-In. We will b e installing a database as a part of this installation in case of any need for other databases being accessed by SharePoint (e.g. Form Based Authentication database)

Step A:

Step B:

Step C:

5. Install SharePoint 2013

We will start installing SharePoint as soon as the SSRS database installation is done. The SQL databases can be installed in parallel with SharePoint installation. We will need the SQL Server ready at the time of running SharePoint Configuration wizard. As a part of SharePoint installation, we will be installing the pre-requisites first and then start installing SharePoint 2013 from the SharePoint Installation screen.

Step A:

Step B:

Step C: Install SharePoint 2013 Server in SharePoint Complete (Farm) Mode

6. Run SharePoint 2013 Configuration Wizard

Step A:

Step B:

Step C:

Step D:

Step E:

7. Configure SharePoint Farm

After the installation and configuration is done, SharePoint administration panel will be available to configure the newly installed SharePoint Farm. As a part of Farm configuration the basic web application will be created with the start-up site collection. If we want to change different service account, we may have to change it from central Administration->Security->Configure Service Accounts. To add or change service accounts, we have to add the Managed Accounts into SharePoint.

8. Install SSRS in SharePoint integrated Mode

To install SSRS in SharePoint Integrated Mode, we have to run the following commands from SharePoint PowerShell.

- Run "Install-SPRSService " cmdlet from "SharePoint 2013 Management Shell" to install the service.

- Run "Install-SPRSServiceProxy" to install the service proxy

- Run "get-spserviceinstance -all |where {$_.TypeName -like "SQL Server Reporting*"} | Start-SPServiceInstance" to start the service.

9. Configure SSRS In SharePoint

Now SSRS is installed in SharePoint. To configure SSRS in SharePoint Integrated Mode, we have to create a SQL Server Reporting Services Service Application and configure System Settings, SSRS Email Settings and Execution Account for the calling services impersonation.

But, if we have SSRS 2008 or 2008 R2, we have to run the SQL Reporting Services Configuration followed by SharePoint Settings for the SSRS. And, this is available under General Application Settings:

Step A:

Step B:

I have tried to elaborate the process by taking step by step snapshot. But, if you face any issues, please feel free to contact me here.

Configure Form Based Authentication in SharePoint 2013

2013-05-31T11:09:00.000-07:00

If we want to Share the SharePoint site among our external vendors, one way of implementing this is to implement the Form based Authentication to the extended SharePoint web application. In this way the vendors will have their LogIn IDs through which they can get access to the SharePoint sites. After the successful installation and configuration of FBA feature in SharePoint, we can create the FBA Users and Roles through a separate web application or as a SharePoint feature. SharePoint 2013 FBA Pack provides the features to Add, Manage FBA users/Roles and also provide you with the web part for Password Change, Request for Access and Custom Login Page with your corporate branding. In this whole process of FBA configuration the following things are involved:

- Create FBA user database or Creating custom provider for User Authentication

- Extend the hosted web application with external Host Header

- Select FBA Authentication in Claim Based Authentication section of Extended Web Application in the configuration page

- Check the "Enable Form Based Authentication (FBA)" and uncheck "NTLM" in Claims Authentication Types Section of Extended Web Application in the configuration page

- Enter the FBA Membership Provider and FBA Role Provider names in Claims Authentication Types Section of Extended Web Application in the configuration page

- Update web.config in three different places (SharePoint Administration, SecurityTokenServiceApplication and Content Web Application)

- Create your custom login page (If you don't want to use the default login page for FBA)

I tried to elaborate the process by taking step by step snapshot. But, if you face any issues, please feel free to contact me here.

1. Create the FBA User Database

2. Grant Appropriate permissions for the newly created database

3. Create a New Web Application and Site Collection in SharePoint (In case if you don't have the base web application created)

4. Extend the web application for Form Based Authentication (FBA): Go to SharePoint Central Administration->Application Management and select the web application that you want to extend. Then click Extend from the Ribbon Toolbar and fill in the required information.

- Extend the hosted web application with external Host Header

- Select FBA Authentication in Claim Based Authentication section of Extended Web Application in the configuration page

- Check the "Enable Form Based Authentication (FBA)" and uncheck "NTLM" in Claims Authentication Types Section of Extended Web Application in the configuration page

- Enter the FBA Membership Provider and FBA Role Provider names in Claims Authentication Types Section of Extended Web Application in the configuration page

5. Update Web.Config for SecurityTokenServiceApplication, SharePoint Administration and Content Web Application

For the SecurityTokenServiceApplication

- In the the config file, find the following sections and add the blue tagged sections. Or add if necessary. But, the connection string section will have to be added at the end of the configuration section which means just before the last line of the web.config file.

<system.web>

<add name="FBAMembershipProvider"

type="System.Web.Security.SqlMembershipProvider, System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"

connectionStringName="FBAUsers"

enablePasswordRetrieval="false"

enablePasswordReset="true"

requiresQuestionAndAnswer="false"

applicationName="/"

requiresUniqueEmail="true"

passwordFormat="Hashed"

maxInvalidPasswordAttempts="5"

minRequiredPasswordLength="7"

minRequiredNonalphanumericCharacters="0"

passwordAttemptWindow="10"

passwordStrengthRegularExpression="" />

</providers>

</membership>

<add name="FBARoleProvider"

type="System.Web.Security.SqlRoleProvider, System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"

connectionStringName="FBAUsers"

applicationName="/" />

</providers>

</roleManager>

</system.web>

<connectionStrings>

<add name="FBAUsers"

connectionString="Data Source=MYSHAREPOINT\SSRSSQL;Initial Catalog=FBAdatabase;Integrated Security=SSPI" />

</connectionStrings>

For SharePoint Central Administration/Content Web Application:

- In the <PeoplePickerWildcards> section (configuration/SharePoint/PeoplePickerWildcards)

Add the following:

<add key="FBAMembershipProvider" value="%" />

- Find the configuration/system.web/membership/providers section

Add the following:

<add name="FBAMembershipProvider"

type="System.Web.Security.SqlMembershipProvider, System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"

connectionStringName="FBAUsers"

enablePasswordRetrieval="false"

enablePasswordReset="true"

requiresQuestionAndAnswer="false"

applicationName="/"

requiresUniqueEmail="true"

passwordFormat="Hashed"

maxInvalidPasswordAttempts="5"

minRequiredPasswordLength="7"

minRequiredNonalphanumericCharacters="0"

passwordAttemptWindow="10"

passwordStrengthRegularExpression="" />

- Find the configuration/system.web/roleManager/providers section

Add the following:

- Find the configuration/connectionStrings section (Or add it if necessary)

Add the following

<connectionStrings>

<add name="FBAUsers" connectionString="Data Source=MYSHAREPOINT\SSRSSQL;Initial Catalog=FBAdatabase;Integrated Security=SSPI" />

</connectionStrings>

6. Install SharePoint FBA Pack to manage FBA Users (http://sharepoint2013fba.codeplex.com/ )

Install (or upgrade) the SharePoint 2013 FBA Pack

· Download and unzip Sharepoint2013FBAPack.X.X.X.zip to the SharePoint server.

· Open PowerShell and navigate to the folder the files were unzipped to.

· Run the following command:

· .\deploy [Site Collection URL]

· e.g. .\deploy http://demo2010a:13824/

· The FBA Pack will be deployed to SharePoint and activated on the specified site collection. If the site collection url is omitted, you will need to manually activate the 'Forms Based Authentication Management' feature in each site collection you wish to use it.

· To uninstall run:

· .\undeploy [Site Collection URL]

SharePoint 2013 Version Comparison Matrix

2013-04-30T10:21:00.001-07:00

Developer features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Access Services	No	No	Yes
App Catalog (SharePoint)	No	Yes	Yes
App Deployment: Autohosted Apps	No	No	No
App Deployment: Cloud-Hosted Apps	No	Yes	Yes
App Deployment: SharePoint-Hosted Apps	No	Yes	Yes
App Management Services	No	Yes	Yes
BCS: Alerts for External Lists	No	Yes	Yes
BCS: App Scoped External Content Types (ECTs)	No	Yes	Yes
BCS: Business Data Webparts	No	Yes	Yes
BCS: External List	Yes	Yes	Yes
BCS: OData connector	No	Yes	Yes
BCS: Profile Pages	No	Yes	Yes
BCS: Rich Client Integration	No	No	Yes
BCS: Secure Store Service	Yes	Yes	Yes
BCS: Tenant-level external data log	No	No	Yes
Browser-based customizations	Yes	Yes	Yes
Client Object Model (OM)	Yes	Yes	Yes
Client-side rendering (CSR)	Yes	Yes	Yes
Custom Site Definitions	No	No	Yes
Custom Site Provisioning	No	No	Yes
Developer Site	No	No	No
Forms Based Applications	No	Yes	Yes
Forms on Spreadsheets	No	Yes	Yes
Full-Trust Solutions	Yes	Yes	Yes
InfoPath Forms Services	No	No	Yes
JavaScript Object Model	Yes	Yes	Yes
List and Library APIs	Yes	Yes	Yes
Remote Event Receiver	Yes	Yes	Yes
REST API	Yes	Yes	Yes
Sandboxed Solutions	Yes	Yes	Yes
SharePoint Design Manager	Yes	Yes	Yes
SharePoint Designer	Yes	Yes	Yes
SharePoint Store	Yes	Yes	Yes
Workflow 2010 (.NET 3.5)	Yes	Yes	Yes
Workflow 2010 (out of the box)	No	Yes	Yes
Workflow 2013	No	Yes	Yes
Workload API: ECM APIs	No	Yes	Yes
Workload API: Search APIs	Yes	Yes	Yes
Workload API: Social APIs	No	Yes	Yes

IT Professional features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Active Directory Synchronization	Yes	Yes	Yes
Alternate Access Mapping (AAM)	Yes	Yes	Yes
Analytics Platform	No	No	Yes
Claims-Based Authentication Support	Yes	Yes	Yes
Configuration Wizards	No	Yes	Yes
Deferred Site Collection upgrade	Yes	Yes	Yes
Distributed Cache	Yes	Yes	Yes
Host Header Site Collections	Yes	Yes	Yes
Improved Permissions Management	Yes	Yes	Yes
Improved Self-Service Site Creation	No	No	Yes
Managed Accounts	Yes	Yes	Yes
Minimal Download Strategy (MDS)	Yes	Yes	Yes
OAuth	No	Yes	Yes
Patch Management	Yes	Yes	Yes
Quota Templates	Yes	Yes	Yes
Read-Only Database Support	Yes	Yes	Yes
Remote Blog Storage	Yes	Yes	Yes
Request Management	Yes	Yes	Yes
Request throttling	Yes	Yes	Yes
Resource throttling	Yes	Yes	Yes
Service Application Platform	Yes	Yes	Yes
SharePoint Health Analyzer	Yes	Yes	Yes
SharePoint Online Admin Center	N/A	N/A	N/A
Shredded Storage	Yes	Yes	Yes
Site Collection Compliance Policies	No	Yes	Yes
Site Collection Health Checks	Yes	Yes	Yes
State Service	Yes	Yes	Yes
Streamlined Central Administration	Yes	Yes	Yes
System Status Notifications	Yes	Yes	Yes
Unattached Content Database Recovery	Yes	Yes	Yes
Upgrade evaluation site collections	Yes	Yes	Yes
Usage Reporting and Logging	Yes	Yes	Yes
Windows PowerShell Support	Yes	Yes	Yes

Content features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Accessibility Standards Support	Yes	Yes	Yes
Asset Library Enhancements/Video Support	Yes	Yes	Yes
Auditing	No	Yes	Yes
Auditing & Reporting (e.g. doc edits, policy edits, deletes)	No	Yes	Yes
Auditing of View Events	No	Yes	Yes
Content Organizer	No	Yes	Yes
Design Manager	Yes	Yes	Yes
Document Sets	No	Yes	Yes
Document Translation in Word Web App	Yes1	Yes1	Yes1
eDiscovery	No	No	Yes
External Sharing: External Access	No	No	No
External Sharing: Guest Link	No	No	No
Folder Sync	No	Yes	Yes
Information Rights Management (IRM)	No	Yes	Yes
In-Place Hold	Yes	Yes	Yes
Managed Metadata Service	No	Yes	Yes
Metadata-driven Navigation	No	Yes	Yes
Multi-stage Disposition	No	Yes	Yes
Office ProPlus (Osub)	No	No	No
Office Web Apps (edit)	Yes1	Yes1	Yes1
Office Web Apps (view)	Yes1	Yes1	Yes1
Office Web Apps Server integration	Yes	Yes	Yes
PowerPoint Automation Services	No	Yes	Yes
Preservation hold library	No	No	Yes
Quick Edit	Yes	Yes	Yes
Related Items	Yes	Yes	Yes
Rich Media Management	No	Yes	Yes
Shared Content Types	No	Yes	Yes
SharePoint Translation Services	Yes	Yes	Yes
Site mailbox	No	Yes2	Yes2
Unique Document IDs	No	Yes	Yes
Video Search	No	No	Yes
WCM: Analytics	No	Yes	Yes
WCM: Catalog	No	No	Yes
WCM: Cross-site publishing	No	No	Yes
WCM: Designer Tools	No	Yes	Yes
WCM: Faceted navigation	No	No	Yes
WCM: Image Renditions	No	No	Yes
WCM: Mobile and Device Rendering	No	Yes	Yes
WCM: Multiple Domains	No	No	Yes
WCM: OOTB Recommendations Webparts	No	Yes	Yes
WCM: Search Engine Optimizations (SEO)	No	Yes	Yes
WCM: Topic Pages	No	No	Yes
Word Automation Services	No	Yes	Yes

Insights features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Business Intelligence Center	No	No	Yes
Calculated Measures and Members	No	No	Yes
Data Connection Library	No	No	Yes
Decoupled PivotTables and PivotCharts	No	No	Yes
Excel Services	No	No	Yes
Field list and Field Support	No	No	Yes
Filter Enhancements	No	No	Yes
Filter Search	No	No	Yes
PerformancePoint Services	No	No	Yes
PerformancePoint Services (PPS) Dashboard Migration	No	No	Yes
Power View	No	No	Yes
PowerPivot	No	No	Yes
Quick Explore	No	No	Yes
Scorecards & Dashboards	No	No	Yes
SQL Server Reporting Services (SSRS) Integrated Mode	No	No	Yes
SQL Server Reporting Services (SSRS) Integrated Mode	No	No	Yes
Timeline Slicer	No	No	Yes
Visio Services	No	No	Yes

Search features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Advanced Content Processing	Yes	Yes	Yes
Content Search Web Part	No	No	Yes
Continuous crawl	Yes	Yes	Yes
Custom entity extraction	No	No	Yes
Deep links	No	Yes	Yes
Event-based relevancy	No	Yes	Yes
Expertise Search	Yes	Yes	Yes
Extensible content processing	No	No	Yes
Graphical refiners	No	Yes	Yes
Hybrid search	Yes	Yes	Yes
Managed navigation	No	Yes	Yes
On-premises search index	N/A	N/A	N/A
Phonetic name matching	Yes	Yes	Yes
Query rules—Add promoted results	No	Yes	Yes
Query rules—advanced actions	No	No	Yes
Query spelling correction	Yes	Yes	Yes
Query suggestions	No	Yes	Yes
Query throttling	No	Yes	Yes
Quick preview	Yes	Yes	Yes
Recommendations	No	Yes	Yes
Refiners	Yes	Yes	Yes
RESTful Query API/Query OM	Yes	Yes	Yes
Result sources	Yes	Yes	Yes
Search connector framework	No	No	No
Search results sorting	Yes	Yes	Yes
Search vertical: "Conversations"	No	Yes	Yes
Search vertical: "People"	No	Yes	Yes
Search vertical: "Video"	No	No	Yes
Tunable Relevancy	No	No	No

Sites features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Change the look	Yes	Yes	Yes
Connections to Microsoft Office Clients	Yes	Yes	Yes
Cross Browser Support	Yes	Yes	Yes
Custom Managed Paths	Yes	Yes	Yes
Governance	Yes	Yes	Yes
Large List Scalability and Management	Yes	Yes	Yes
Mobile Connectivity	Yes	Yes	Yes
Multi-Lingual User Interface	Yes	Yes	Yes
My Tasks	Yes	Yes	Yes
OOTB Web Parts	Yes	Yes	Yes
Permissions Management	Yes	Yes	Yes
Project functionality for team sites	Yes	Yes	Yes
Project site template	Yes	Yes	Yes
Project Summary web part	Yes	Yes	Yes
Project workspace	Yes	Yes	Yes
Public Website (SPO)	N/A	N/A	N/A
SharePoint Lists	Yes	Yes	Yes
SharePoint Ribbon	Yes	Yes	Yes
Task list	Yes	Yes	Yes
Team Site: Drag & Drop	Yes	Yes	Yes
Team Site: Notebook	Yes	Yes	Yes
Team Site: Simplified Access	Yes	Yes	Yes
Templates	Yes	Yes	Yes
Themes	Yes	Yes	Yes
Work Management Service	Yes	Yes	Yes
Usage Analytics	Yes	Yes	Yes

Social features	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Ask Me About	No	Yes	Yes
Blogs	Yes	Yes	Yes
Communities Reputation, Badging, and Moderation	No	Yes	Yes
Community	Yes	Yes	Yes
Company Feed	Yes	Yes	Yes
Follow	No	Yes	Yes
Microblogging	No	Yes	Yes
Newsfeed	No	Yes	Yes
One Click Sharing	No	Yes	Yes
People, Sites, Document Recommendations	No	Yes	Yes
Personal Site	No	Yes	Yes
Photos and Presence	Yes	Yes	Yes
Profile	No	Yes	Yes
Ratings	No	Yes	Yes
Site Feed	Yes	Yes	Yes
Skydrive Pro	Yes	Yes	Yes
Tag profiles	No	Yes	Yes
Tasks integrated with Outlook	No	Yes	No
Trending Tags	No	Yes	Yes
Wikis	No	Yes	Yes

Add-Ons	SharePoint Foundation	SharePoint Server 2013—Standard Edition	SharePoint Server 2013—Enterprise Edition
Additional Storage	N/A	N/A	N/A
App Hosting: Base	N/A	N/A	N/A
App Hosting: Per User	N/A	N/A	N/A
Azure Provisioned Apps: Access Services	N/A	N/A	N/A
Azure Provisioned Apps: Custom Code in Azure LWR	N/A	N/A	N/A
Duet Online	N/A	N/A	N/A

Installation and Configuration of PowerPivot in SharePoint 2010

2013-01-10T10:37:00.001-08:00

PowerPivot is a Business Intelligence reporting feature of SharePoint. PowerPivot helps analyze the data in more efficient way. It is highly interactive and user self-service reporting tool. Using PowerPivot we can analyze data from SQL Server Analysis services or any other OLAP data mart. Now let us see how we can configure the PowerPivot for SharePoint. To install and configure PowerPivot, we will need the following things hands:

1. SharePoint Service account for PowerPivot and Excel Services. If we use the same service account, the whole process will be easy.

2. SQL Server 2008 R2 Enterprise Edition Installer

Installation of PowerPivot:

To start with the installation, the first thing we have to do is to create a configuration file like this:

1. Create a text file in notepad and paste the following paragraph into there:

<?xml version="1.0" encoding="utf-8" ?>

</runtime>

</configuration>

2. Save it as Setup100.exe.config in %ProgramFIles%\Microsoft SQL Server\100\Setup Bootstrap\SQLServer2008R2\x64. We have to do this to work around a known bug in "Analysis Service Installation for SharePoint" (Ref: http://support.microsoft.com/kb/2261507)

3. Start the SQL Server Enterprise Edition installer as "Administrator"

4. Select "New Installation or add features to an existing installation"

5. Install setup support files

6. Select "New Installation or add shared features" in "Installation Type"

7. Select "SQL Server PowerPivot for SharePoint" and select "Existing Farm" in the "Add PivotTable for SharePoint to:"

8. In the "Instance Configuration" leave the default name of the Database Instance which is "POWERPIVOT" and click "Next"

9. Add the Service Account/Password in "Server Configuration" area for the PivotTable database instance

10. Add SharePoint Administrator account to "Analysis Service Configuration"

11. Then finish the installation by clicking "Next" and finally "Finish" and you will see the successful installation message from the installer

Configuration of PowerPivot:

1. Configure and Start the following services:

a. Excel Calculation Service

b. Secure Store Service

c. Claims to Windows token Service

d. SQL Server Analysis Services

e. SQL Server PowerPivot System Service

To do that, you may have to go to SharePoint central Administration->System Settings->Manage Services on Server

2. Create PowerPivot Service Application:

To create the PowerPivot Service Application, you have to go to Central Administration->Application Management->Manage Service Applications (In Service Applications group). Then type the new application pool name with the Configurable Identity that will be used to connect to external data sources to create the report. Leave the other selections (Name, Database Server, and database name) as-is or change it based on your preferences. Finally select the checkbox for "Add the proxy for this PowerPivot service application to the default proxy group".

3. Now we have to create the Target Application ID for PowerPivot in "Secure Store Service". To do that, please go to SharePoint Central Administration –> Application Management –> Manage Service Applications->Secure Store Service. If you haven't created the "Secure Store Service" yet, please follow another of my blog entry on PerformancePoint (http://fazlulchowdhury.blogspot.ca/2012/12/performancepoint-ultimate-bi-reporting.html ).

Then click on "New" from the Ribbon and fill in the Target Application ID, Display name and Administrator Email and click "Next". Leave everything as-is and click "Next". Add "target Application Administrators" and click ok to save the configuration. It will take you back to "Secure Store Service Application" section.

At this point, you have to set the credentials for the Secure Store Service Application for PowerPivot that you have just created. Check the checkbox beside the target Application ID and click on the "Set Credentials" button from the ribbon. The set the following:

a) Credential Owner: This is the owner of the credential. It will be used by PowerPivot unattended account but this is not the account used as the unattended account. Generally it is SharePoint Administrator Service account.

b) Windows user name and password: This is the actual account used as the unattended account. Generally it is SharePoint Service account for Pivot Table or Excel Services.

Click OK to save the configuration.

Then associate the application with PowerPivot Service Application. To do that, go to SharePoint Central Administration –> Application Management –> Manage Service Applications -> PowerPivotServiceApplication1 and click on Manage from the ribbon or just click it. It will open PowerPivot dashboard. Click on Configure Service Application Settings in Actions frame. It opens PowerPivot Settings. Find PowerPivot Unattended Data Refresh Account input box under Data Refresh. Type the ID of the application created in Secure Store Service (i.e. PowerPivotDataRefresh).

4. Set the Excel Services properly. To do that Go to SharePoint Central Administration –> Application Management –> Manage Service Applications and select Excel Services Application->Trusted File Locations. Then click on "http://" and change the values as required but uncheck the "Warn on Refresh" and Select Trusted data connection libraries and embedded under Allow External Data.

5. Deploy the PowerPivot Solutions: If you go to SharePoint Central Administration->System Settings->Manage Farm Solutions, you will notice that there are two new .wsp files:

powerpivotfarm.wsp and powerpivotwebapp.wsp. powerpivotfarm.wsp has been globally deployed. But, you have to deploy powerpivotwebapp.wsp to the web application where you are going to use PowerPivot. Click the powerpivotwebapp.wsp file and select "Deploy" followed by selecting right web application to deploy.

Now the configuration is complete for "SharePoint PowerPivot Services". In case of getting errors (i.e. "Cannot Open Excel Workbook" ) with PowerPivot, please make sure that the "Unattended Service Account" for Excel Services and PowerPivot Services has access to the SharePoint Database and connecting data source as well. To allow the service account access to SharePoint, please run the following script for each account.

Add-PSSnapin Microsoft.SharePoint.PowerShell -EA 0

$webApp = Get-SPWebApplication "---SharePoint Site URL---"

$webApp.GrantAccessToProcessIdentity("---Unattended Service Account----")

Testing PowerPivot in a Business Intelligence enabled site:

1. Go to Site Collection Administration-> Go to top level site settings. Then go to Site Collection Administration-> Site collection features and activate following services:

a) PerformancePoint Services Site Collection Features

b) PowerPivot Feature Integration for Site Collections

c) SharePoint Server Publishing Infrastructure

Also make sure that "PerformancePoint Services Site Features" is activated under "Manage Site Features"

2. Create a Business Intelligence Template Site: Go to Site Actions->View All Site Content->Create. Then go to Data under "All Categories" and select "Business intelligence Center" and create the site.

3. Create a Pivot table Gallery: Go to Site Actions->View All Site Content->Create.

4. Start creating Pivot table reports

PerformancePoint - Ultimate BI Reporting Tool for SharePoint

2012-12-18T10:27:00.001-08:00

PerformancePoint is a Business Intelligence reporting feature of SharePoint. To start working with PerformancePoint, we have to configure it correctly and make sure that all the required services are started and correctly configured. Let us see which services are needed for PerformancePoint.

Services:

1. PerformancePoint Service

2. Secure Store Service

To View and Enable the services, we have to go to SharePoint Administration->Application management->Service Applications (Manage services on server)

Now, let me guide you through the configuration process of the above mentioned services.

To do that, we have to go to Application management->Service Applications (Manage service applications)

Then, find out is the service applications are created or not. If not created then create the service applications by selecting "PerformancePoint Service Application" or "Secure Store Service" from "New" Toolbar. If these are already been created then click on the service applications to configure.

1. Creating Secure Store Service Application: To Create a New "Secure store Service", we have to type a name for the application followed by entering the "Database Server"/"Database Name"/Authentication. This is basically the SharePoint Database Server with a new database name. Select "Windows Authentication" and create a new application pool with "Configurable" service account. Then provide this account access to the newly created database.

2. Configuring Secure Store service: To configure Secure Store Service, we need to generate a Key for "Unattended Account". So, we will click on "Generate New key" button from the toolbar.

It will ask for "Pass Phrase" and "Confirm Pass Phrase" to Re-encrypt the Database. After that you will see that you will need to create a "Secure Store target Application" in this newly created "Store Service Application".

3. Creating PerformancePoint Service Application: To Create a New "PerformancePoint Service Application", we have to type a name for the application followed by entering the "Database Server"/"Database Name"/Authentication. This is basically the SharePoint Database Server with a new database name. Select "Windows Authentication" and create a new application pool with "Configurable" service account. Then provide this account access to the newly created database.

4. Configure PerformancePoint Service Application: Once the service application is created, we have to create an unattended service account so that we can use that account to access an external data source. Let us click on "Manage" button from toolbar.

Then select "PerformancePoint Service Application Settings"

And, select the "Unattended Service Account"->User name and Password. And, set other configuration variable as required otherwise keep them as is.

5. After the service applications are created or configured, we have to make sure that the services and service applications are in "Started" state.

6. Once the service applications are configured, it can be consumed to create PerformancePoint Reports, Dashboards, KPI Reports or even Scorecards.

7. Turn On "PerformancePoint Services Site Collection Feature" in "Site Collection Administration" (Site Settings->Site collection Administration->Site Collection Features).

8. Turn On "PerformancePoint Services Site Features" in "Site Settings" (Site Settings->Site Actions->Manage Site Features).

9. After activating required features, we can consume the PerformancePoint service. We have to create a "Data Connections Library" (let us call it as PPS Connection) for reports and a "Performance Point Content List" (let us call it as PPS Configuration)

10. To start working with PerformancePoint Designer click on "Add New Item" in "PPS Configuration" list. It will connect to the designer and open it up like this:

Then right click on the PPS Connection to create the database connection from the database. If we are creating the report based on data cube, we can select the SQL Analysis Cube right from this screen.

After creating the database connection, create the PerformancePoint report by right clicking on "PPS Configuration" and selecting appropriate type:

After that we should be able to see the report to host inside SharePoint as an excellent BI reporting platform which has drill down and decomposition tree as well.

Import/Export operation of a particular SharePoint site using Power Shell

2012-11-30T14:37:00.001-08:00

The easiest way to copy SharePoint sites cross domain is to export the site using PowerShell and Import it into destination SharePoint using another PowerShell script. There are couple of things we have to keep in mind though:

1. It's better to import into a brand new site

2. Install all associated features/web parts even before running the PowerShell script, otherwise the import will fail

3. Recreate the permissions and security layers after succesful import

4. Fix all the absolute Links URL referenced inside the application

5. Rebuild calendar Overlays (If existed)

6. Fix BCS (Business Connectivity Services) based External Content Types and re-create or re-point them to appropriate data sources

7. Fix Alerts

8. Re-visit workflows and event-handlers and Re-Install if needed. because, workflows are not portable

9. Fix all the broken Client scripts (Java-Script or JQuery)

Now let's see how to export and import from SharePoint "A" site to SharePoint "B" site.

1. Run the following powershell command in SharePoint "A" site

Export-SPWeb -Identity http:\\ServerName:port\Site -Path c:\\backup\TeamSiteBack.dat

2. Run the following powershell command in SharePoint "B" site

Import-SPWeb -Identity http:\\ServerName:port\Site -Path c:\\backup\TeamSiteBack.dat -Force

SharePoint Architecture for Globally Hosted Solution

2012-11-29T09:55:00.001-08:00

GLOBAL SOLUTIONS FOR SHAREPOINT

- Central solution

- Central with regional sites solution

- Distributed solution

- Comparison of supported solutions

- Accommodating organizational scenarios

Term	Definition
Central site	The location that hosts the majority of corporate data and/or employee computers connected by using a local area network (LAN). Corporate servers at the central site are typically contained within a data center.
Regional site	A location that hosts a subset of corporate data and/or employee computers connected together by using a combination of LAN and wide area network (WAN) links. A regional site is connected to a central corporate site by using a WAN link. - A site located in a different region or country. - A site separated by insufficient network links (even if the site is within reasonable proximity to the central site). - A site physically separated from the central site but otherwise well-connected (such as a government agency or branch office).

CENTRAL SOLUTION

In the central solution, all Office SharePoint Server components and portal site farm services are hosted at the central site. Users access all services across the WAN. In the following illustration, the central site is hosted in Madrid. Users at regional sites in Los Angeles, South Africa, and Hong Kong access sites and services from the central site in Madrid over the WAN.

Design

Central site hosts	Regional sites host
Server farm running Office SharePoint, including: - Published intranet content - Team or division collaboration sites - My Sites - Enterprise-wide search	Nothing

Guidance

- The central solution typically works well when work sites and users span a single continent

- If users are unable to use the sites and services because performance over the WAN is too slow, consider deploying multiple server farms to regional sites. However, before scaling beyond the central solution, evaluate the ways in which we can optimize a central solution to improve performance over the WAN.

Disadvantages

- Service application data cannot be isolated

- Individual departments or teams cannot manage service applications on their own

Recommendation

- The central solution is the recommended solution for environments in which bandwidths and latencies between WAN connections provide a reasonable user experience. This is the easiest of the three supported solutions to maintain and operate.

CENTRAL WITH REGIONAL SITES SOLUTION
The central with regional sites solution includes one centralized site plus regional sites where Office SharePoint Server is running at each of the regional sites. Users collaborate with their local teams on the local server farm. Users access all enterprise-wide features across the WAN. The Trusted My Site Host Locations feature is implemented to ensure that users are directed to the Shared Services Provider (SSP) intended for their profile. In the following illustration, the central site is hosted in Madrid. Server farms running Office SharePoint Server are also deployed at each of the regional sites in Los Angeles, South Africa, and Hong Kong. The bi-directional arrows show that collaboration takes place across regional sites in addition to collaboration with the central site.

Design

Central site hosts

Regional sites host

Server farm running Office SharePoint Server, including:

- Published intranet content.

- Enterprise-wide search (optional).

- My Site redirection (Trusted My Site Host Location). This ensures that a user coming from a regional farm is redirected to the My Site on the farm where their profile resides.

- Profile synchronization (optional).

- Aggregation of content developed at regional sites (optional).

Server farm running Office SharePoint Server, including:

- Team or division collaboration sites.

- My Sites with My Site redirection. This ensures that a user coming from the central farm or another regional farm is redirected to the My Site where their profile resides.

- Search for local content.

Guidance

- The central with regional sites solution provides local access to services that regional users are likely to use most — for example, collaboration sites and My Sites. It also provides local search for local content. However, this solution increases the complexity and operations costs of the overall solution. This solution also requires greater organizational coordination to build effective governance of content that is authored in multiple geographic locations.

DISTRIBUTED SOLUTION

The distributed solution includes a number of regional sites, each of which operates autonomously from other regional sites. This solution includes a centralized portal site, but the solution is not optimized for collaboration across regional sites. In the following illustration, the central site is hosted in Madrid. Regional sites are hosted at various cities around the world. Each regional site is connected only to the central site.

Design

Central site hosts

Regional sites host

Server farm running Office SharePoint Server, including:

Published intranet content.
Aggregation of content developed at regional (for example, for reporting or for records retention).
My Site redirection and profile synchronization if Office SharePoint Server is used at the regional sites (optional).

Either:

Server farm running Office SharePoint Server, including:

Team or division collaboration sites.
My Sites with My Site redirection.
Search for local content.
Read-only copy of published intranet content (optional).

Server farm running Windows SharePoint Services 3.0 including:

Team or division collaboration sites.
Windows SharePoint Services search scoped to the site collection.

Guidance

- The distributed solution is recommended for organizations that have many offices distributed geographically, such as branch offices. This solution provides local access to collaboration sites at the regional office. It also provides the ability to share content between a regional site and the central site. For example, the central site can publish content to regional sites for local access. Regional sites can upload content to the central site to consolidate content or to share regional data.

- Because of the number of regional sites, it might not be practical to provide enterprise-wide search in which the central site crawls content on each of the regional sites. However, in a distributed model in which the regional sites are largely autonomous, you can provide users access to content beyond their regional site in the following two ways:

Implement a governance strategy in which regional sites upload or publish content relevant to the entire organization to the central site. Enable regional users to connect to the central site to search on this content.
Publish or upload targeted content from the central site to each of the regional sites. Users can search on this content by using their local search service.

COMPARISON

Item	Central solution	Central with regional sites solution	Distributed solution
Overall recommendations	Preferred if the available network bandwidth across the WAN provides a satisfactory experience for regional users. Advantages include: No redundant hardware or services. Least complex administration. Consistent experience for regional users (search, My Sites, alerts, and profiles).	Use only if the available network bandwidth does not support the use of portal site services and features over the WAN or if the user experience is unacceptable for most SharePoint site use. Performance of collaboration within each regional site is greatly improved. Collaboration across the central and regional sites is possible, but performance depends on the WAN links.	Use if the available network bandwidth does not support the use of a central solution and there are many regional sites. This solution does not provide collaboration for teams working across geographical sites. However, this solution greatly increases the performance of collaboration sites within each regional site.
Administration	Centralized — all server hardware and server farm services are hosted at the central site.	Requires regional management of the server farm and services.	Requires regional management of the server farm and services.
WAN bandwidth	All content must be transferred over the WAN. The user experience is dependent on the available bandwidth. However, the bandwidth load is generated only by the regional users. The central server farm search services do not crawl data at the regional sites, which reduces bandwidth requirements.	Collaboration within regional farms does not affect WAN communication. Profile synchronization uses WAN connections to synchronize profile information across SSPs in all regional farms. If the regional site chooses to use enterprise-wide search services, network bandwidth across the WAN is required to send and receive results to search queries and to download content associated with search queries. Additionally, if the regional site chooses to be included in enterprise-wide search scopes, the central site search service uses a significant amount of bandwidth to crawl the regional content. Search can consume all network resources on the WAN link when in operation over that link.	Collaboration within regional farms does not affect WAN communication. Profile synchronization uses WAN connections to synchronize profile information across SSPs in all regional farms. Content sharing between regional sites and the central site is subject to WAN connections. Because of the number of regional sites, it might not be practical to provide enterprise-wide search in which the central site crawls content on each of the regional sites. However, if the regional site chooses to use enterprise-wide search services, network bandwidth across the WAN is required to send and receive results to search queries and to download content associated with search queries.
Capacity planning	The greatest consideration when planning for capacity at the central site is the available network bandwidth to the regional site.	Capacity of the regional server farm can be calculated by using the standard capacity planning guidelines. If the central site is providing enterprise-wide search to regional sites, be sure to calculate this additional load for the central farm.	Capacity of the regional server farm can be calculated by using the standard capacity planning guidelines. If the central site is providing enterprise-wide search to regional sites, be sure to calculate this additional load for the central farm.
Search	Search is hosted entirely at the central site, either by the central farm or by a farm dedicated to hosting shared services. The central search service provides enterprise-wide search. Search administrators at the central farm can also create custom search scopes that are specific to each region. This provides a way for regional users to search only on content that is hosted at their local site.	The regional site can use any (or none) of the following search options: Regional search hosted by the regional server farm. Enterprise-wide search hosted by the central site. This can be configured to include or not include content from the regional farm. If the enterprise-wide search includes content from the regional farm, WAN bandwidth is used to crawl the regional farm content. Regional search scopes hosted by the central site (requires adequate WAN links to crawl the regional content). Federated search	The regional site provides regional search. Because of the number of regional sites, it might not be practical to provide enterprise-wide search in which the central site crawls content on each of the regional sites. However, users at the regional site can search content at the central site. The central site can also publish selected content to regional sites so this content is available locally for local search. Federated search is also a good option for providing search access to targeted content.
Site directory (optional)	Some site templates include a site directory. You can use a site directory to list sites that are relevant for specific regional offices.	Team sites on regional farms can be connected to your central farm by manually listing these sites in a site directory. Targeted sites on the central farm can be listed in the site directory of a regional site.	Team sites on regional farms can be connected to your central farm by manually listing these sites in a site directory. Targeted sites on the central farm can be listed in the site directory of a regional site.
Crawling content sources	A single SSP is recommended for providing enterprise-wide search. Implementing more than one SSP does not allow relevancy to be applied across all of the content.	To provide enterprise-wide search, add a content source for each region. Increase the change log retention time on Web applications on regional farms to coordinate with the incremental crawl schedule.	To provide enterprise-wide search, add a content source for each region. Increase the change log retention time on Web applications on regional farms to coordinate with the incremental crawl schedule.
My Sites	A single SSP can host My Sites for the organization.	Host My Sites at the region closest to users. Use the Trusted My Site Host Locations feature to ensure that users create only one My Site in the SSP intended for their use.	Host My Sites at the region closest to users. Use the Trusted My Site Host Locations feature to ensure that users create only one My Site in the SSP intended for their use.
User profiles	User profiles are managed at the central site only.	To replicate profile information between the central and regional farms, implement the profile replication tool.	To replicate profile information between the central and regional farms, implement the profile replication tool.
Globalization	Sites or site collections can be created in different languages by using language packs. Additionally, site variations allow site owners to publish content translated in different languages in the same site collection. However, the Central Administration site and Shared Services Administration sites are provided in a single language.	Same as central solution.	Same as central solution.
Backup and recovery	All backup and recovery procedures are completed at the central site.	Performing backup and recovery procedures across the WAN is not recommended. Backup and recovery tasks must be completed at regional sites.	Performing backup and recovery procedures across the WAN is not recommended. Backup and recovery tasks must be completed at regional sites.

SHAREPOINT FARM ARCHITECTURES

- Single farm, single service application group

- Single farm, multiple service application groups

- Enterprise services farms

- Cross-organization farms

Service Applications

Service applications that are typically deployed for dedicated use by a specific team or department include:

- Excel Services To optimize performance for a targeted team or to isolate sensitive data

- Managed Metadata To allow a team or department to manage their own taxonomy, hierarchies, keywords, and so on. SharePoint Server 2010 combines the results of multiple Managed Metadata service applications, so taxonomies, content types, and other elements can be shared across an organization

- Business Data Connectivity Individual teams or departments can integrate with their own line-of-business data systems and keep the data isolated from the rest of the organization.

SINGLE FARM, SINGLE SERVICE APPLICATION GROUP

Architecture

- In an architecture that includes a single farm and a single service application group, the default service application group is used for all Web applications in the farm. All sites have access to all of the service applications that are deployed in the farm.

Design Diagram

Advantages

- It is the simplest architecture to deploy

- All service applications are available to all Web applications

- Farm resources are used most efficiently

- All service applications are managed centrally

Disadvantages

- Service application data cannot be isolated

- Individual departments or teams cannot manage service applications on their own

Recommendations

- The architecture that includes a single farm and a single service application–group is the recommended configuration for most organizations, at least initially. This configuration works well when you want to host many sites for a single company on the same farm. Use this configuration to meet the following goals:

- We want to optimize the resources required to run service applications within a farm

- We are sharing content and profile data across sites that otherwise require process isolation for performance or security reasons.

SINGLE FARM, SINGLE SERVICE APPLICATION GROUP

Architecture

- If teams require dedicated service applications, build an architecture by using one or more custom groups of service applications. Follow these guidelines:

- Deploy specific service applications for dedicated use by one or more teams within an organization

- Ensure that the dedicated service applications are not also included in the default group

- Create one or more Web applications that use a custom group of service applications. The SharePoint administrator selects the service applications that are included in the custom group.

Design Diagrams

Advantages

- They accommodate multiple organizational goals in the same farm

- Service data can be isolated

- Individual teams or departments can manage the service applications that have been dedicated for their use

- Sites can be configured to use a subset of service applications.

Disadvantages

- They are more complex to configure and manage

- Farm resources are consumed to support multiple instances of some service applications, which can affect performance

Recommendations

- Architectures that include multiple service application groups work well for companies that have divisions or teams that require dedicated service applications or isolated service data, or for sites that are set up with a narrower scope, such as partner collaboration

- Additionally, when multiple groups of service applications are configured, teams and sites can consume services that are offered enterprise-wide, such as profile and search services, while at the same time isolating the use of targeted services for security or performance reasons

ENTERPRISE SERVICES FARMS

Architecture

- An enterprise services farm is a server farm that is dedicated to hosting service applications for an organization. The following diagram shows an enterprise services farm that hosts the most frequently deployed cross-farm service applications. It also shows several common kinds of farms that consume services from an enterprise services farm

Design Diagram

PUBLISHED CONTENT-ONLY FARMS (all service applications are remote)/(FARM 2)
Architecture

- We can deploy a server farm without deploying any service applications locally. In Farm 2, no service applications are hosted locally. All service applications are consumed from a separate farm. This configuration works well for content that is published. It reduces the administrative efforts required to host a published content farm and allows an organization to take advantage of centrally managed service applications.

- Use this configuration when we have the following goals:

- We want to optimize the resources within a farm for hosting content, instead of running service applications

- We are integrating with organization-wide profiles, metadata, search, and other centrally managed resources

COLLABORATION FARMS (mix of local and remote service applications)/(Farm3)
Architecture

- We can deploy a server farm without deploying any service applications locally. In Farm 2, no service applications are hosted locally. All service applications are consumed from a separate farm. This configuration works well for content that is published. It reduces the administrative efforts required to host a published content farm and allows an organization to take advantage of centrally managed service applications.

Deciding Factor

- Use this configuration when we have the following goals:

- We want to optimize the resources within a farm for hosting content, instead of running service applications

- We are integrating with organization-wide profiles, metadata, search, and other centrally managed resources

FARM FOR SPECIALIZED DEPARTMENTS (mix of local and remote service applications)/(Farm4)

Architecture

- Some teams within an organization might require a separate deployment of specific services for the following reasons:

- To ensure data isolation (such as Business Data Connectivity data)

- To provide the ability to autonomously manage service applications (such as Managed Metadata)

Characteristics

- The characteristics of this farm include the following:

- It consumes centrally managed service applications that include Managed Metadata

- It also includes its own Managed Metadata service application, so this team can autonomously manage its own metadata. Because this service application is shared, the metadata from the rest of the organization can be integrated with this metadata

Deciding Factor

- Use this configuration to meet the following goals:

- Allow a specialized team or department to manage metadata on their own

- Ensure that specific service data is isolated and managed separately from the rest of the organization

CROSS ORGANIZATIONAL FARMS
Architecture

- Service applications can be shared across any farm, not only enterprise services farms. Consider sharing service applications across farms in the following scenarios

Scenario A

- To provide enterprise-wide service applications without a dedicated enterprise services farm, as shown in the following diagram

Scenario B

- To share resources across farms and to avoid deploying redundant service applications, as shown in the following diagram.

Ref: Supported global solutions for Office SharePoint Server

-->

SharePoint User Profile Service Account Priviledges

2012-11-01T15:10:00.001-07:00

SharePoint Profile Services need to have Active Directory Replication Changes privilege to sync Active Directory with SharePoint. So, Replicate Directory Changes privilege needs to be granted to the SharePoint Profile Service account that is being used for User Profile Services Sync.

Steps:

1. Go to the Active Directory User and Computer console.

2. Right Click on Domain.Com and choose Delegate Control

3. Select the Service Account

4. Select Create a Custom task to delegate.

5. Select delegate for folders and existing objects.

6. Delegate active directory replication with general permissions.

7. Click next and Finish

Then go to SharePoint Administration->System Settings->Manage Services on Server and Stop/Start User Profile Service and User Profile Synchronization Service….

Architecting SharePoint 2010 Infrastructure

2012-10-25T08:51:00.001-07:00

Overview

This document presents such deployment-related information as the different deployment stages and environments, plus a flowchart that illustrates the steps for installing and configuring SharePoint 2010 Products.

Requirements

Before we can deploy Microsoft® SharePoint® Server 2010 or Microsoft® SharePoint® Foundation 2010, we must meet the following requirements:

Hardware requirement: 64-bit

SharePoint Server 2010 and SharePoint Foundation 2010 are 64-bit applications and can only run on a 64-bit edition of the Windows Server® 2008 operating system. We must have hardware that supports the use of a 64-bit operating system and 64-bit SQL Server.

Hardware requirements—Web servers, application servers, and single server installations

The requirements in the following table apply both to installations on a single server with a built-in database and to servers running SharePoint Server 2010 in a multiple server farm installation.

Component

Minimum requirement

Processor

64-bit, four cores

RAM

·         4 GB for developer or evaluation use

·         8 GB for production use in a single server or multiple server farm

Hard disk

80 GB for system drive

We must have sufficient space for the base installation and sufficient space for diagnostics such as logging, debugging, creating memory dumps, and so on. For production use, we also need additional free disk space for day-to-day operations. Maintain twice as much free space as we have RAM for production environments.

Hardware requirements—Database servers

The requirements in the following table apply to database servers in production environments with multiple servers in the farm.

Component

Minimum requirement

Processor

64-bit, four cores

RAM

·         64-bit, four cores for small deployments

·         64-bit, eight cores for medium deployments

Hard disk

80 GB for system drive

We must have sufficient space for the base installation and sufficient space for diagnostics such as logging, debugging, creating memory dumps, and so on. For production use, we also need additional free disk space for day-to-day operations. Maintain twice as much free space as we have RAM for production environments.

Operating system requirement: Windows Server 2008 or Windows Server 2008 R2

SharePoint Server 2010 and SharePoint Foundation 2010 must be run on a 64-bit edition of Windows Server 2008 with Service Pack 2 or Windows Server 2008 R2.

Database requirement: 64-bit SQL Server 2005 SP2 or 64-bit SQL Server 2008

For server farm installations of SharePoint Server 2010 and SharePoint Foundation 2010, we must be running 64-bit versions of Microsoft SQL Server® 2005 or Microsoft SQL Server 2008 (each with the appropriate service packs and updates) on our database servers.

Server Categories for SharePoint Farm (Three Tear)

A SharePoint farm consists of three tiers out of which Web or Front-End tear consists of SharePoint Web Servers that hosts SharePoint applications, Application Tear that hosts SharePoint services and Search components and finally Database Tear that hosts SharePoint databases. The following illustration shows a SharePoint Server 2010 farm with two front-end Web servers (Web-1 and Web-2) that serve content and host the search query component. The only application server (App-1) hosts Central Administration and the search crawl component for the farm.

Domain Controller

SharePoint needs a pre-defined domain controller to setup and configure SharePoint Farm Accounts or configure directory synchronization with SharePoint profile services. Gibson will be using iTechPlanet.com as the core Domain which is already pre-configured. It is recommended to use the following symbol for domain controllers.

Database Servers

SharePoint farm uses Microsoft SQL server to store SharePoint databases. If not pre-installing databases, the configuration database is created when we run the SharePoint Products 2010 Configuration Wizard. All other databases are created when we create the service applications and Web applications that require them. It is recommended to use the following symbol for database servers.

Web Servers

SharePoint web servers are basically front end web application servers that hosts SharePoint connected through NLBs (Network Load Balancer). The fundamental role of a front-end Web server is hosting the Web pages, the Web services, and the Web Parts that are necessary to process requests received from users. The Web server directs these requests to the application server, which returns the results to the front-end Web server. Depending on farm requirements, the front-end Web server may also be configured to support search in scenarios where there are not any dedicated search servers. It is recommended to use the following symbol for domain controllers.

Application Servers

SharePoint application servers host most basic SharePoint services, SharePoint central administration and Search services. It is recommended to use the following symbol for domain controllers.

Prepare and Build

Here is the brief overview of different types of Servers required for SharePoint Farm and how to install and build them.

Deployment stage and environment

Planning:

Purpose:

Before we can deploy, we must plan the solution we want to deploy. After the planning stage, we move through the deployment stages in the following table, updating and revising wer plans as we test.

Steps:

·         Perform business analysis

·         Determine goals and objectives

·         Determine infrastructure requirements

Output:

·         Solution plan

·         Topology and resource requirements

Proof of Concepts (POC):

Purpose:

Used for determining whether a solution will meet business needs and to determine an appropriate infrastructure plan.

Steps:

·         Deploy farm

·         Deploy solution

·         Collect benchmark data

·         Evaluate proof of concept

·         Refine goals and infrastructure requirements

Output:

·         Updated solution plan

·         Updated topology and resource requirements

Characteristics:

POCs are often created during evaluation phase to test the product or solution. A POC can be hosted on a development environment or on a small scale production computer. Sometimes the POC moves through several stages in continuous development.

Expected Duration and Stability:

·         Set up, reviewed, and then discarded

·         Not a stable environment – no service-level agreement for users

Farm Architecture:

Medium scale

Two tiers Application/Web servers

and Database server

Small scale

All server roles installed on a

single physical server

Pilot (small scale test):

Purpose:

Pilots are used to test a solution on a small scale. Pilots are used to test solution readiness (no real data, just functionality testing). A pilot can also be used to test for production characteristics (real data, actual work); this is recommended.

Steps:

·         Deploy pilot farm

·         Deploy pilot solution

·         Collect benchmark data

·         Evaluate pilot

·         Refine goals and infrastructure requirements

·         Determine operations plan

Output:

·         Updated solution plan

·         Updated topology and resource requirements

·         Operations plan

Characteristics:

A limited set of users has access to the environment to test the functionality and performance of the solution and infrastructure.

Expected Duration and Stability:

·         Limited time frame

·         Limited service-level agreement for solution readiness testing

·         Production service-level agreement for production testing

Farm Architecture:

Medium scale

Closer to planned production environment.

Small scale

Each server role on its own server.

Development:

Purpose:

Development phase is being used for developing applications and solutions for SharePoint 2010 Products.

Steps:

·         Deploy development computers or farm

·         Develop solution

·         Test and evaluate solution

·         Refine solution

Output:

·         Solution

Characteristics:

Development environments exist throughout the lifecycle of the project. Initial development of the solution is followed by testing and refining the solution. Note that SharePoint Workspaces can be used to keep development environments synchronized.

Expected Duration:

·         Exists throughout solution lifecycle

·         Not a stable environment – no service-level agreement for users

Farm Architecture:

Medium-large scale

Closer to planned production environment.

Small scale

Each server role on its own server.

User Acceptance Test (UAT):

Purpose:

A pre-production environment used for testing solutions against a subset or complete copy of production data. Also used for validating the backups or operational procedures.

Steps:

·         Deploy UAT farm

·         Deploy UAT solution

·         Implement operations plan

·         Evaluate solution

·         Evaluate operations plan

·         Test for capacity and performance

Output:

·         Updated operations plan

Characteristics:

Topology should be as similar to production environment as possible. Testers ensure that all solution elements function as expected in network and security conditions that match the conditions of the production environment.

Expected Duration:

·         Long-term availability – this is a stable pre-production environment

·         Limited service-level agreement for users

Farm Architecture:

Medium-large scale

Closer to planned production environment.

Small scale

Each server role on its own server.

Production:

Purpose:

This is the live environment that our users interact with in production environment.

Steps:

·         Deploy production farm

·         Deploy production solution

·         Implement operations plan

·         Deploy additional environments:

·         Authoring and staging farms

·         Geo-distributed farms

·         Services farms (Search, Taxonomy)

Output:

·         Updated production plan

Characteristics:

It is business critical and appropriate service-level agreements are in place.

Expected Duration:

·         Long-term availability – this should be a stable environment

·         Full service-level agreement for users, appropriate to the solution and business requirements

Farm Architecture:

Medium-large scale

Closer to planned production environment.

Small scale

Each server role on its own server.

Configure Settings, Services, Solutions and Sites

In this phase, we prepare the farm to host our site content by configuring global settings, creating services applications, deploying customizations, and creating and populating the sites. We can use the Farm Configuration Wizard to these configuration steps, or we can perform them by using either the SharePoint Central Administration Web site or Windows PowerShell. Configuration steps are not isolated to a specific tier in the server infrastructure. These steps include only a high-level overview of the process. To successfully plan and perform our deployment, follow the recommendations and procedures in the Planning Guide and the Deployment Guide for SharePoint Server 2010 or SharePoint Foundation 2010.

Best practices for maintaining multiple environments

Keep the environments synchronized by using the following:

·         Profile replication engine – a tool for keeping profile and social data synchronized across farms.

·         Content deployment – a method of moving content between authoring, staging, and live environments.

·         Mirroring and log shipping – two techniques for keeping content changes synchronized across farms.

Keep the environments clean by using the following best practices:

Be sure to reformat your computers before re-using hardware between environments or within an environment. Do not simply uninstall and reinstall. If there are old customizations or configurations, they can affect how that computer works and introduce errors into your environment.

Service Accounts

SharePoint needs some pre-defined service accounts to install and configure SharePoint. SharePoint will elevate the following accounts as necessary during installation. So, for each environment we will be adding "_Environment" for each service account. Like:

For Sand Box environment "sp_admin" will be "sp_admin_SB"

For Development environment "sp_admin" will be "sp_admin_DEV"

For Test environment "sp_admin" will be "sp_admin_TST"

For Production environment "sp_admin" will be "sp_admin_PRD"

Account name

Role

Domain rights

Local SharePoint Server rights needed

SQL rights needed

sp_admin

Used to install SharePoint binaries.

Domain User

Local administrator on all SharePoint boxes

dbcreator and securityadmin SQL roles

sp_farm

Farm account. Used for Windows Timer Service, Central Admin and User Profile service

Domain User

Local Admin during UPS provisioning, log on locally right

None

sp_webapp

App pool id for content web apps

Domain User

None

None

sp_serviceapps

Service app pool id

Domain User

None

None, unless using Office Web Apps. Them must give access to content databases manually

sp_search

Search process id

Domain User

None

None

sp_content

Account used to crawl content

Domain User

None

None

sp_userprofile

Account used by the User Profile services to access Active Directory

Must have Replicating Change permissions to AD. Must be given in BOTH ADUC and ADSIEDIT. If domain is Windows 2003 or early, must also be a member of the "Pre-Windows 2000" built-in group.

None

None

sp_superuser

Cache account

Domain User

Web application Policy Full Control

Web application super account setting

None

sp_superreader

Cache account

Domain User

Web application Policy Full read

Web application super reader account setting

None

DNS Names

It is recommended to use the DNS names (http://spsandbox) instead of physical server address and port number (http:// CADVHYP001:80) for each web application in a SharePoint Environment. The DNS names are being used to assign the host headers for the web applications.

Environment

Web Application

DNS Name

Sandbox

Intranet Team Sites

http://intranetSB

Development

Intranet Team Sites

http://intranetDEV

Test

Intranet Team Sites

http://intranetTST

Production

Intranet Team Sites

http://intranet

Sandbox

Intranet Publishing Site

http://TheInnovationSB

Development

Intranet Publishing Site

http://TheInnovationDEV

Test

Intranet Publishing Site

http://TheInnovationTST

Production

Intranet Publishing Site

http://TheInnovation

Sandbox

Internet Publishing Site

http://TechPlanetSB

Development

Internet Publishing Site

http:// TechPlanetDEV

Test

Internet Publishing Site

http:// TechPlanetTST

Production

Internet Publishing Site

http://www.TechPlanetCanada.com

References:

http://technet.microsoft.com/en-us/library/cc850692.aspx

http://technet.microsoft.com/en-us/library/cc263199.aspx

Advanced Software Testing - What does that mean?

2012-07-18T12:01:00.001-07:00

The term “software quality” can be interpreted in many ways, often depending on one’s perspective. For example, a developer may consider the quality of software to be dependent on the design and maintainability of the code. On the other hand, an end-user may consider speed and reliability to be indicators of high-quality software. In general, though, the perceptions of quality are rooted in how well software conforms to its requirements. Software requirements, when adequately defined, describe how software should behave functionally (how to respond to particular input) and non-functionally (maintainability, security, performance, etc.).

Well tested software decreases the possibility of failure at a greater extent and improves the quality and dependability at a greater extent. Software Testing may eventually reduce Errors, Failures, Faults or Incidents. There are different types of testing that can be initiated to cover different areas of a software system. The testing path for the Software include (Unit Tests, Integration Tests, Functionality Tests, UI Tests, Usability Tests, and User Acceptance Tests etc.).

The requirements themselves were derived from the accompanying documentation and were functional in nature. Thus, we are measuring the quality of the units--methods, in this case--by how well they adhere to the specified requirements. Here, we tried to cover basic Unit testing scenarios.

To do that we have created a Test Suite which has numerous test cases that exercises different units being revised by correction and repairing faults caused by errors. The test cases also find the failures that was caused by faults and eventually caused by specific errors. Test cases may have many test stubs and test drivers. Software testing needs rigorous test planning at the first place. Which may include: Analysis Software Model/Requirements and estimate expected results. Then deriving effective test cases followed by executing tests and compare results with expected results. At the end of test suite generation there is a need to verify the coverage of the test suite.

Our strategy, at its core, is driven by the requirements specified in the documentation. We do not want to make assumptions, either based on how we think the methods should behave, or how we were observe them behaving. In essence, we’re focused on testing how well the methods conform to their specifications.

Basic Testing Definitions:

Errors: Mistakes that are being initiated by people (There were no Database Cluster environment)
Fault: Fault is the result of an Error (Database is offline or down)
Failure: Failure occurs when fault executes (Web system cannot communicate with the database)
Incident: Incidents are basically consequences of failure (I cannot log into my bank account)

Goals of Software Testing:

According to Dijkstra, 1972 - “Program testing can be used to show the presence of bugs, but never to show their absence”. The main goal of software testing is to have dependable software on which we can gain only sufficient confidence. Which eventually means, the software may be faulty even we try to cover all bits and pieces of the SUT (System Under Test) in out testing procedure.

Testing Process Overview:

Derive Test cases from Software Representations
Estimate the expected result for each test case (Oracle)
Execute the test cases
Get test results
Compare Test Result with Expected Result

Testing should be effective, repeatable, systematic, automated and should help locate faults for debugging the program. Dependability of software systems may have several aspects (Correctness, Reliability, Availability, Safety, Robustness or may be combination of those). We need to determine when to stop testing by having sufficient confidence.

Black and White Box Testing:

Black-box testing is a testing process through which the specifications of the software system are being tested where as White-box testing is for the Implementation of the software solution. Missing functionality cannot be revealed by white-box testing whereas unexpected functionality cannot be revealed by black box testing.

There are different techniques for black box testing:

Equivalence Class Partitioning: partitions of the input set in which input data have the same effect on the program. By completeness we mean entire input set is covered. By WECT we mean, choosing one variable value form each equivalence class (one ai, bi, and ci) such that all classes are covered. By SECT we mean the Cartesian product of the partition subsets.
Boundary-Value Analysis: Testing the boundary values of input space. Min, Min+, Max, Max- are some boundary values for testing
Category-Partition: The system is divided into several “functions” that can be tested. Categories are the major properties or characteristics for each parameter.
Decision tables: It helps to express test requirements in a usable form.
Cause-Effect Graphs: It is a graphical technique that helps derive decision tables
Logic Functions: It maps from n Boolean input variables to 1 Boolean output variable
Black-box Test Criteria: In Black-box testing criteria there exist number of test coverage criteria:

                   Predicate coverage
                   Clause coverage
                   Combinatorial coverage
                   (in)active clause coverage

According to Marick, the following things need to be addressed to have a large scale testing:

To test every functionality, generate Black-box tests from requirements
Check the code coverage after the functional tests are all verified.
In case of imperfect structural coverage, generate functional tests for additional coverage.

There are different techniques for white box testing as well:

Control flow coverage

Statement/Node Coverage (Covering all nodes in CFG)

Edge Coverage (Select a test set T such that, by executing P for each test case t in T, each edge of P’s control flow graph is traversed at least once)

Condition Coverage (Select a test set T such that, by executing P for each element in T, each edge of P’s control flow graph is traversed, and all possible values of the constituents of compound conditions (defined below) are exercised at least once)

Path Coverage (Select a test set T such that, by executing P for each test case t in T, all paths leading from the initial to the final node of P’s control flow graph are traversed)

Data flow coverage: Assignment of values to variables and their uses
Analyzing coverage data
Mutation Testing: It is typically Fault based testing. Faults are being injected to monitor different behavior (these faults versions are called mutants). Then kill the mutants one by one. By killing the mutants, we mean if the test cases detect the differences then the mutants are killed and test is considered adequate if all the mutants get killed by the test cases.
Integration testing
Generating test data
Tools

Ref: Dr. Vahid Garousi

Export/Import Active Directory Entries To/From a Domain Controller

2012-07-09T15:14:00.000-07:00

We can use the ldifde.exe to import and export Active Directory entries to/from a Domain Controller. Ldifde.exe can be found in c:\windows\system32 folder. If you can’t find it then run the add-ons to install that command line tool.

To export the Organizational Units or simply OUs, please execute the following command:

ldifde -f exportOu.ldf -s dcdomnp01 -d "dc=MyDomain,dc=Com" -p subtree -r "(objectCategory=organizationalUnit)" -l "cn,objectclass,ou"

To export User Accounts please execute the following command:

ldifde -f Exportuser.ldf -s dcdomnp01 -d " dc=MyDomain,dc=Com" -p subtree -r "(&(objectCategory=person) (objectClass=User)(givenname=*))" -l

"cn,givenName,objectclass,samAccountName,company,c,co,employeeNumber,givenName,sn,streetAddress,l, st,postalCode,title,costCenter,department,manager,"

And, to import them into the new domain, please execute the following commands:

ldifde -i -f ExportOU.ldf -s MyToDomain.Com

ldifde -i -f Exportuser.ldf -s MyToDomain.Com

To Add/Remove the AD attributes please change the attribute section on the previously stated commands.

Done….It’s that simple

Calendar Overlays in SharePoint

2012-04-10T07:07:00.001-07:00

“Calendar Overlay” is one of the most popular features of SharePoint. By using Calendar overlays, we can actually view multiple calendars into one. Let’s say the CEO wants to view the calendars of his departments/divisions, we can use calendar overlays to view the calendars of each department/division into a merged calendar. There is one limitation though. In SharePoint 2010, we can have up to 10 calendars while creating the overlaying calendar. Let us now see how can we do the Calendar Overlays.

First create a basic calendar list (CEO View) where we will be overlaying other calendars that are already created.

Then click on Calendars Overlay button up top.

Click on New Calendar

Let’s say call the first overlay calendar “Area 1”. And, then select the type as “SharePoint”. You can also overlay Exchange calendars. But, we will be concentrating on SharePoint calendars for the time being. Type the web url where your other calendars are created/hosted (typically same url). Then click “Resolve” to Get the Calendar lists and associated views to choose from. If you want the filtered calendar in overlay view, you may want to create a filtered view for that specific calendar beforehand. After selecting the appropriate calendar and view, click on “Always Show” and click OK. In this way create/link other calendars to overlay into the “CEO View”.

So, we have created a single calendar which is basically the merged view of several different calendars for different divisions/departments. And, each calendar will show with different colored tags as follows:

Done !!!

How to Import/Export Custom Content Types in SharePoint 2010 using Powershell

2012-02-08T10:30:00.001-08:00

Let us first create two files of which the file names and content will be as follows:

1. File Name: CTypes.ps1. This file contains the script to export existing site content types (From: http://apps-d/myApp and Content types group: My Custom Content Types ) into an XML script (To: D:\Development\SharePoint\MYAPP\Scripts\Script-SiteContentTypes.xml)

$sourceWeb = Get-SPWeb http://apps-d/myApp
$xmlFilePath = "C:\Script-SiteContentTypes.xml"

#Create Export File
New-Item $xmlFilePath -type file -force

#Export Content Types to XML file
Add-Content $xmlFilePath ""
Add-Content $xmlFilePath "`n"
$sourceWeb.ContentTypes ForEach-Object {
if ($_.Group -eq "My Custom Content Types") {
Add-Content $xmlFilePath $_.SchemaXml
}
}
Add-Content $xmlFilePath ""

$sourceWeb.Dispose()

2. File Name: Import.ps1. This file contains the script to import the content types into http://206137l/MyApp from XML script (To: C:\Script-SiteContentTypes.xml)

$destWeb = Get-SPWeb http://206137l/MyApp
$xmlFilePath = "D:\Development\SharePoint\MYAPP\Scripts\Script-SiteContentTypes.xml"
#Create Site Content Types
$ctsXML = [xml](Get-Content($xmlFilePath))
$ctsXML.ContentTypes.ContentType ForEach-Object {

#Create Content Type object inheriting from parent
$spContentType = New-Object Microsoft.SharePoint.SPContentType ($_.ID,$destWeb.ContentTypes,$_.Name)

#Set Content Type description and group
$spContentType.Description = $_.Description
$spContentType.Group = $_.Group

$_.Fields.Field ForEach-Object {
if(!$spContentType.FieldLinks[$_.DisplayName])
{
#Create a field link for the Content Type by getting an existing column
$spFieldLink = New-Object Microsoft.SharePoint.SPFieldLink ($destWeb.Fields[$_.DisplayName])

#Check to see if column should be Optional, Required or Hidden
if ($_.Required -eq "TRUE") {$spFieldLink.Required = $true}
if ($_.Hidden -eq "TRUE") {$spFieldLink.Hidden = $true}

#Add column to Content Type
$spContentType.FieldLinks.Add($spFieldLink)
}
}

#Create Content Type on the site and update Content Type object
$ct = $destWeb.ContentTypes.Add($spContentType)
$spContentType.Update()
write-host "Content type" $ct.Name "has been created"
}

$destWeb.Dispose()

3. Then open up SharePoint 2010 Management Shell in Administrative mode and change the directory to the script folder (i.e. D:\Development\SharePoint\MYAPP\Scripts). Then, run those scripts together with the folder location (i.e. D:\Development\SharePoint\MYAPP\Scripts\ CTypes.ps1 and D:\Development\SharePoint\MYAPP\Scripts\ Import.ps1)

Then, if we go to the new site, we will see that the content types have been imported successfully.

Installing SharePoint 2010 in Windows 7

2012-02-06T09:33:00.000-08:00

Installing SharePoint 2010 in Windows 7 is recommended for development environment and not intended for a full blown production SharePoint Farm. Installing SharePoint 2010 in windows 7 may be little bit tricky and this is why I wanted to put those issues and fixes together in my blog. Let us start the installation process.

1. Preconfigure your Windows 7 host system: As a hardware recommendation 4GB should be ok and 8GB will be better if you want to use Visual Studio and SharePoint Designer working on the same machine and running side-by-side. Install the WCF Hotfix for Microsoft Windows. The hotfix is available in Windows Server 2008 R2 and Windows 7. This hotfix also updates the ASP.NET common language runtime (CLR) to recognize a new option of optimizing compilations. To take advantage of this option, set the optimize Compilations property of the <compilation> tag in your web.config file to true: <compilation optimizeCompilations="true">. For Windows 7, the standalone installation is recommended but you can also use a separate SQL Server instance for SharePoint configuration database.

2. Install MS SQL Server 2008 Patch (If you have SQL 2008 Express Installed): If you are using a local instance of Microsoft SQL Server 2008, install the Microsoft SQL Server 2008 KB 970315 x64 (SQL Server 2008 KB 970315 x64). If your development environment uses a remote instance of Microsoft SQL Server 2008 or if it has a pre-existing installation of Microsoft SQL Server 2008 on which KB 970315 x64 has already been applied, this step is not necessary.

3. Install the Prerequisites for SharePoint 2010:

(a) Copy the SharePoint Foundation installer files into your Local folder

(b) Update the files\Setup\config.xml fileis configuration tag as <Setting Id="AllowWindowsClientInstall" Value="True"/> and Save. If you do not fix the config file, you may get the following error message:

(d) Manually enable each of the required Windows Features. You can do this quickly by copying and running the following command in a Command Prompt window (Administrative Mode – Make sure the following lines are in single line and without line breaks):

start /w pkgmgr /iu:IIS-WebServerRole;IIS-WebServer;IIS-CommonHttpFeatures;IIS-StaticContent;IIS-DefaultDocument;IIS-DirectoryBrowsing;IIS-HttpErrors;IIS-ApplicationDevelopment;IIS-ASPNET;IIS-NetFxExtensibility;IIS-ISAPIExtensions;IIS-ISAPIFilter;IIS-HealthAndDiagnostics;IIS-HttpLogging;IIS-LoggingLibraries;IIS-RequestMonitor;IIS-HttpTracing;IIS-CustomLogging;IIS-ManagementScriptingTools;IIS-Security;IIS-BasicAuthentication;IIS-WindowsAuthentication;IIS-DigestAuthentication;IIS-RequestFiltering;IIS-Performance;IIS-HttpCompressionStatic;IIS-HttpCompressionDynamic;IIS-WebServerManagementTools;IIS-ManagementConsole;IIS-IIS6ManagementCompatibility;IIS-Metabase;IIS-WMICompatibility;WAS-WindowsActivationService;WAS-ProcessModel;WAS-NetFxEnvironment;WAS-ConfigurationAPI;WCF-HTTP-Activation;WCF-NonHTTP-Activation

(e) Verify that the required Windows Features are enabled. The command in the previous step enables all of the required features in the Internet Information Services section of the Windows Features dialog box (which you can access through the Programs section in Control Panel). Check that you have enabled all of the required Windows Features. If any features are missing in your operating system, return to the Internet Information Services section of the Windows Features dialog box and enable them. To do that, you have to go to Control Panel and Click on Programs and then click on Turn Windows Features On or Off as the following screen:

(f) Then Expand Internet Information Services and make sure that the following features are selected:

(g) Restart the PC to apply the changes

4. Install the Prerequisites for SharePoint 2010: Now, we have all the pre-requisites installed and all the required windows features are turned on. Now we can begin the real installation of SharePoint foundation. To Install SharePoint 2010, let us run the setup.exe in Administrative mode and choose standalone install. In case of error, always visit log files from the given link else run the configuration wizard at the end of SharePoint 2010 Foundation install.

5. Bug Fixing: If you get the following errors during the configuration process, please fix as follows:

(a) Error: An exception of type System.IO.FileNotFoundException was thrown. Additional exception information: Could not load file or assembly 'Microsoft.IdentityModel, Version 3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified.

Fix: Install Windows Identity Foundation pack from: http://www.microsoft.com/downloads/details.aspx?FamilyID=eb9c345f-e830-40b8-a5fe-ae7a864c4d76&displaylang=en#filelist

(b) Error: An exception oftype Microsoft.SharePoint.Upgrade.SPUpgradeException was thrown. Additional exception information: Failed to call GetTypes on assembly Microsoft.Office.Server.Search, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c. Could not load file or assembly 'System.Web.DataVisualition, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

Fix: Install Chart Controls from: http://go.microsoft.com/fwlink/?LinkID=122517.

(c) Error: Failed to create the configuration database.An exception of the type Microsoft.SharePoint.SPException was thrown. Additional exception information: User cannot be found

Fix: Make sure that the logged-in user has sufficient rights into the SQL Server to create the configuration database.

(d) If you get more error, please visit the following site: http://msdn.microsoft.com/en-us/library/ee554869(office.14).aspx

Microsoft Dynamics CRM - Working With New Versions of Existing Workflows

2011-10-26T14:22:00.001-07:00

Update a workflow for a Release by editing existing and saving as new name in DEV. Set previous workflow versions to Draft status, which CRM will ignore. Repeat steps 1.1 to 1.2.7 above and add/remove/update workflow steps to match new process flow.

Note: Export from PROD back to DEV if necessary to ensure same initial version, but only un-publish PROD workflows if during scheduled outage. The reason for un-publishing a workflow prior to export is that imports to different environment may error out in certain scenarios if trying to import a Published workflow.

Microsoft Dynamics CRM - Maintaining Existing Workflows Already in PROD

2011-10-26T14:21:00.001-07:00

1. Open the workflow that you want to copy.

2. Un-publish the workflow (careful: if the workflow is in production, you may want to make sure that all open instances of the workflow have completed running and that the workflow will not be needed for the next few steps - best to do this between the hours of 1:00 am and 3:00 am )

3. Convert the workflow to a template. This is accomplished by opening the unpublished workflow, and selecting "Workflow Template" for the field titled "Publish As"

4. Publish the template: Before you can use the template, you have to publish it. You publish a template in the same manner that you publish a normal workflow (i.e. click the "Publish" button).

5. Create a new workflow. Use the template you just created to create this new workflow. If you wanted to keep your old workflow running while you develop a new one, then go ahead and publish this new one immediately - and then create another new workflow that you will use to make your changes.

Microsoft Dynamics CRM - Create, Test, Deploy a Workflow

2011-10-26T14:16:00.000-07:00

1.1.1. Add new workflow in DEV/TST

1.1.2. MS CRM User Interface represents Workflow in serialized format. Add each step and label each with corresponding step number in Process Flow Diagram.

1.1.3. Publish Workflow in DEV/TST and test each scenario as admin first, then as users with appropriate security roles. Business to confirm each scenario ends as process flow indicates.

1.1.4. After testing completed successfully and agreed to by the business, temporarily un-publish workflow for Export.

1.1.5. Export CRM Customizations (or just the Workflow) and check into Source Control.

1.1.6. Publish workflow again after Export complete to resume functionality in DEV/TST.

1.1.7. Steps to deploy on Release Outage:

1.1.7.1. Import Customizations from Source Control into PROD Environment

1.1.7.2. Go To Settings > Workflows and open (double click) on new workflow

1.1.7.3. Edit each child step where ‘red circle with white x’ appears, you must remove references to lookups from prior environment. Update the step by referring to values from PROD environment (re-associate to lookup of same value) and save.

1.1.7.4. When all errors eliminated, Save and Close workflow.

1.1.7.5. Publish Workflow.

Fazlul's Information and Communication Technology Blog

Data Science, Big Data & Microsoft Machine Learning

SAP RFC Integration through BizTalk

Cloud Computing Technology Assessment

Table Of Contents

Preface

Introduction

Definition of Cloud Computing

Cloud Computing Service Models

Cloud Computing Deployment Models

Why Cloud Computing?

The Benefits of Cloud Computing

Survey Results on Cloud Computing

SharePoint Server 2013 Infrastructure Standard

SharePoint BI - How to fix SSRS Issues

SharePoint Hosting Solution - Which One to Choose? - Part 1

Vision Statement:

Enterprise Goals:

Primary Goals:

Some Research Findings:

SharePoint Hosting Solutions:

Some Key Features and Considerations while choosing the right SharePoint Hosting Solution:

Conclusion

How SharePoint Can Benefit Your Enterprise

SharePoint 2013 across multiple servers for a three-tier farm (Traditional Topology)

How to install SharePoint 2013 with SSRS 2012 Integrated Mode

Configure Form Based Authentication in SharePoint 2013

For the SecurityTokenServiceApplication

SharePoint 2013 Version Comparison Matrix

Installation and Configuration of PowerPivot in SharePoint 2010

PerformancePoint - Ultimate BI Reporting Tool for SharePoint

Import/Export operation of a particular SharePoint site using Power Shell

SharePoint Architecture for Globally Hosted Solution

GLOBAL SOLUTIONS FOR SHAREPOINT

- Central solution

- Central with regional sites solution

- Distributed solution

- Comparison of supported solutions

- Accommodating organizational scenarios

CENTRAL SOLUTION

Design

Guidance

- The central solution typically works well when work sites and users span a single continent

Disadvantages

- Service application data cannot be isolated

- Individual departments or teams cannot manage service applications on their own

Recommendation

- The central solution is the recommended solution for environments in which bandwidths and latencies between WAN connections provide a reasonable user experience. This is the easiest of the three supported solutions to maintain and operate.

Design

Guidance

Design

Guidance

COMPARISON

SHAREPOINT FARM ARCHITECTURES

- Single farm, single service application group

- Single farm, multiple service application groups

- Enterprise services farms

- Cross-organization farms

Service Applications

Service applications that are typically deployed for dedicated use by a specific team or department include:

- Excel Services To optimize performance for a targeted team or to isolate sensitive data

- Managed Metadata To allow a team or department to manage their own taxonomy, hierarchies, keywords, and so on. SharePoint Server 2010 combines the results of multiple Managed Metadata service applications, so taxonomies, content types, and other elements can be shared across an organization

- Business Data Connectivity Individual teams or departments can integrate with their own line-of-business data systems and keep the data isolated from the rest of the organization.

SINGLE FARM, SINGLE SERVICE APPLICATION GROUP

Architecture

- In an architecture that includes a single farm and a single service application group, the default service application group is used for all Web applications in the farm. All sites have access to all of the service applications that are deployed in the farm.

Design Diagram

Advantages

- It is the simplest architecture to deploy

- All service applications are available to all Web applications

- Farm resources are used most efficiently

- All service applications are managed centrally

Disadvantages

- Service application data cannot be isolated

- Individual departments or teams cannot manage service applications on their own

Recommendations

- We want to optimize the resources required to run service applications within a farm

- We are sharing content and profile data across sites that otherwise require process isolation for performance or security reasons.

SINGLE FARM, SINGLE SERVICE APPLICATION GROUP

Architecture

PUBLISHED CONTENT-ONLY FARMS (all service applications are remote)/(FARM 2)
Architecture

COLLABORATION FARMS (mix of local and remote service applications)/(Farm3)
Architecture

CROSS ORGANIZATIONAL FARMS
Architecture