Hi everyone, I am Quentin Christensen and I work on document and records management functionality for SharePoint. Electronic discovery (commonly referred to as eDiscovery) is an area we are supporting with new set of capabilities in SharePoint Server 2010. In case you are not familiar with eDiscovery, it is the process of finding, preserving, analyzing and producing content in electronic formats as required by litigation or investigations. eDiscovery is an important concern for all of our customers and given that SharePoint has grown to be an integral part of collaboration, document, and records management for many organizations, we recognize the need to support the eDiscovery process for SharePoint content.

Microsoft Office SharePoint Server 2007 included a hold feature that could be used for eDiscovery, but it was scoped to the Records Center site template. With SharePoint Server 2010 the eDiscovery capabilities have been greatly expanded to provide more functionality and the power to use these features across your entire SharePoint deployment.

In this post, I want to highlight three major improvements in SharePoint that support eDiscovery. You can:

• Manage holds and conduct eDiscovery searches on any site collection
• Use SharePoint Server Search or FAST Search for SharePoint out of box to search and process content
• Automatically copy eDiscovery search results to a separate repository for further analysis

Read on to learn how SharePoint Server 2010 can support your eDiscovery initiatives and provide you with the tools you need to manage holds, identify, and collect SharePoint content.

The eDiscovery Process

The Electronic Discovery Reference Model from EDRM (edrm.net) provides an overview of the different parts of the eDiscovery process:

SharePoint Sever 2010 addresses the Information Management, Identification, Preservation and Collection stages. While this blog post will focus mostly on the identification, preservation and collection components, SharePoint provides a rich Information Management platform for Collaboration, Social Computing, Document Management and Records Management.  This means that you can take a proactive approach to eDiscovery by putting a governance framework in place and using appropriate disposition policies to expire content. Managing content and deleting it when it is no longer needed will reduce the amount of content that must be indexed and searched, and collected for eDiscovery.  The result is that eDiscovery costs can be dramatically reduced, changing the problem from finding a needle in a hay stack to finding a needle in a hay bale. Ultimately, the key to achieving legal compliance for eDiscovery obligations is built upon a foundation of robust Information Management.

When an eDiscovery event occurs, such as a receipt of complaint, discovery, or notice of potential legal claim, the identification stage begins. Content that may be subject to eDiscovery must be identified and searches are conducted to find that content. That content needs to be preserved and at some point, the content will be collected.

The eDiscovery Features

Hold and eDiscovery

Hold and eDiscovery is a site level feature that can be activated on any site.

Activating this feature creates a new category in Site Settings that provides links to Holds and Hold Reports lists. There is also a page to discover and hold content that allows you to search for content and add it to a hold. Once the Hold and eDiscovery feature is activated you can create holds and add to hold any content in the site collection. By default only Site Collection administrators have access to the Hold and eDiscovery pages. To give other users permission, add them to the permissions list for the Hold Reports and Holds lists. This will also give access to the Discover and hold content page.

You can manually locate content in SharePoint and add it to a hold, or you can search for content and add the search results to a hold. With the Hold and eDiscovery feature you can create holds in the hold list and then manually add content to the relevant hold by clicking on Compliance Details from the drop down menu for individual items.

Then click on the link to Add/Remove from hold.

And you can select the relevant hold to add to or remove from.

By manually adding an item to hold you will block editing and deletion of that item until it is released from hold. You will notice that the document now has a lock icon showing that it cannot be edited or deleted.

Each night a report for each hold is generated by a timer job. If you need a hold report faster you can manually run the Hold Processing and Reporting timer job in Central Administration.

Search and Process

You can manually add items to hold on any site collection, which is great. But that doesn’t help you find the content you don’t already know about. What if you have a large amount of items you want to find and add to a hold? For that you can use the features on the Discover and hold content page, which is a settings page in Site Settings. From this page you can specify a search query and then preview the results. The configured search service (SharePoint Search Server or FAST Search for SharePoint) will automatically be used. You can then select the option to keep items on hold in place so they cannot be edited or deleted, or if you have configured a Content Organizer Send to location in Central Administration you can have content copied to another site and placed on hold. You may want to create a separate records center site for a particular hold to store all content related to that hold. The Content Organizer is a new SharePoint Server 2010 feature based on the Microsoft Office SharePoint Server 2007 Document Router with richer functionality to automatically classify content based on Content Type or metadata properties. Look for a future blog post covering the Content Organizer.

Holding content in place is recommended if you want to leave content in the location is was created with all the rich context that SharePoint provides, while blocking deletion and editing of content. Be aware that this will prevent users from modifying items. If you prefer users to continue editing documents, then use the copy to another location approach.

When searching and processing, the search will by default be scoped to the entire Site Collection and run with elevated permissions so all content can be discovered. The search can be scoped to specific sites and you can also preview search results before adding the results to a hold. Items can be placed on multiple holds and compliance details will show all of the holds that are applied to an item.

In summary, SharePoint Server 2010 contains key features that make it an essential aspect of your eDiscovery strategy. With the new SharePoint Server 2010 capabilities you can easily apply proper retention policies for all content and make it easier to discover content if an eDiscovery event occurs. eDiscovery often prescribes tight deadlines for production. SharePoint 2010 helps you find the right content and deliver it faster.

Quentin Christensen
Program Manager – Document and Records Management
Microsoft

Hi everyone. It’s Adam here again – this time I want to talk to you today about another key area of the content management world: Document Management (DM). Over the next few months, you’ll be hearing from several members of the engineering team about new DM features that help you get the most value out of your document corpus. We’ll also discuss how key early adopters of SharePoint 2010 used new DM features to solve the toughest information governance challenges.

Today, though, I’d like to spend time talking about what the team has learned about the document management space since SharePoint 2007 and take you on a journey through the key tenets that guided our DM vision this release.

Recap: Document Management in SharePoint 2007

SharePoint 2007 was the first release where SharePoint really broke out of its collaboration role and enabled customers to apply structure and management to their document libraries. A lot of the key DM infrastructure was established in that release: Check in/Check Out, Major/Minor Versioning, Per-Item Permissions, Content Types, Workflows, and the Recycle Bin are just a few examples. Of course, all of these features tightly integrated with the Office client applications such as Word, Excel, and PowerPoint to make it simple for end users to interact with the document repository (a core design tenet that carries through in 2010).

Features like these enabled customers to start creating high-value knowledge repositories on SharePoint 2007.

For 2010, we looked to build off of 2007’s gigantic success, and we rallied our designs around three key ideas:

Tenet #1: Manage the unmanaged

As we looked at how our customers were starting to use the 2007 system’s DM features, we noticed an interesting trend: These features were not just part of managed document repository deployments. Indeed, the traditional DM features were getting heavy usage in average collaborative team sites as well. Customers were using them to apply policy and structure as well as gather insights from what otherwise would have been fairly unmanaged places. SharePoint was being using to pull more and more typically unstructured silos into the ECM world.

This is a key insight that really drove our investments in SharePoint 2010. For instance, one of our key new features in SharePoint 2010 is the notion of a Document Set. You can think of a document set as a “folder on steroids.” It allows you to group related documents together so that they share metadata and have a common homepage, workflows, and archival process:

The Welcome Page of a document set is a customizable page that allows users to discover the content in the set, view and sync metadata between items in the set, and manage the set.

When it came time to design this feature, we knew people would want to use it to manage very structured and rigid official processes (e.g. a pharmaceutical company submitting forms to a regulatory agency). But equally important to us was that the feature can be used in a lightweight team site to manage most processes that requires multiple documents to be bound together (e.g. a team that just needs to put together a pitch book/sales proposal that includes a PowerPoint deck, a spreadsheet of costs, and a document that describes the sales pitch).

Enabling the document set feature to be used informally and easily is one way we are expanding the value of ECM in the minds of SharePoint end users.

Tenet #2: Social computing and enterprise metadata are game changers.

As we started to design out the DM feature set for this release, we quickly realized the power of metadata – both structured taxonomies as well as lightweight folksonomies (keywords) – as transformative forces in the document management space. A SharePoint 2010 document repository would need to take full advantage of both concepts.

There are two key principles that enable SharePoint 2010 users to take advantage of metadata. First is on the tagging side: it’s easy for a site to use enterprise wide content types and taxonomies and it’s also simple for a user to tag with them.

SharePoint 2010 offers consistent management of metadata that any SharePoint site can hook in to with virtually no effort. This allows the entire enterprise to be talking the same language. Tangibly, you can do things such as define the list of products you sell once and have that data available in any SharePoint site.

Note how the type-ahead functionality makes it easy for a user to pick a value from this folksonomy. Also note how the West Coast tag was automatically filled out for the user because it was set as the default value for all documents in this library.

The second key principle is how SharePoint takes advantage of these tags. For instance, a SharePoint 2010 document library can be configured to use metadata as a primary navigation pivot. You can think of metadata based navigation as a virtual folder structure that can be used to filter the items in the library:

Instead of navigating by traditional folders, a user filtered the library to the virtual folder that contains just sales materials about Contoso’s tent products.

It’s a virtuous cycle here: Easy metadata entry allows items to be tagged, which can drive navigation. And because users need the metadata to navigate the repository, this incentivizes them to tag the items!

Tenant #3: The browser as a powerful document management application.

SharePoint has always been used for many scenarios, but perhaps it’s known best for two things:

· A best of breed tool for creating web pages and sites

· A place to store, manage, and collaborate on documents

SharePoint 2010 makes a big bet that creating a knowledge management repository requires the merger of both of these worlds. The browser is increasingly becoming the key technology for information workers – both inside the corporate firewall and on the consumer front. Sure, people will always want to download documents to take with them – but they also want to use the browser to interact with the document and see a wealth of context about the document (e.g. metadata, related documents, wiki pages about the document’s topic).

It’s time for the industry to expect any document management system to also be great at creating pages or wikis that add context to the documents’ content. And any system that doesn’t is going to start looking antiquated.

SharePoint 2010 delivers on this vision in a few different ways. First, if you’ve installed the Office Web Apps (licensed as part of the Office 2010 suite), the default click for a document library can be configured to load Office documents in the browser:

Without ever leaving the browser, users can quickly view Office documents stored in SharePoint.

Second, we spent a lot of time this release thinking about how the web content management features can be used in document repositories. For instance, the ever popular Content Query web part can be used to roll up all the documents related to a particular topic:

A content steward might create a page about a particular topic (e.g. a new product). This page includes text about the product, marketing pictures, as well as roll ups of all the documents tagged with the product.

This vision allows you to combine two very powerful aspects of SharePoint into one solution to your organization’s knowledge discovery problem. It’s a merger of an enterprise wiki and a traditional enterprise document repository.

Wrapping up: A lot more to come!

I hope this post gives some context on where we are going with document management in SharePoint 2010 and beyond. Feature wise, we really only hit a few of the many DM features that make up SharePoint 2010 – stay tuned for future posts as we deep dive into a lot more! And feel free to leave comments about what you’d like us to blog about (especially if you’ve downloaded the Beta and given SharePoint 2010 a test drive already!)

Thanks for reading.

Adam Harmetz

Lead Program Manager, Document and Records Management

My name is Jim Masson, and I’m the Group Program Manager for the Enterprise Content Management team within SharePoint. My team is part of the engineering team, and is responsible for designing the features around content management, including managing documents, web content, rich media assets, records, and a new service for managing shared content types and taxonomy.

With the coming launch of SharePoint 2010, this seemed like a good time to ramp up the ECM team blog, and start a conversation about the SharePoint 2010 release. In the lead up to the offical launch of the product and beyond, various members of the team will be posting details about the major feature areas and features within ECM in SharePoint 2010, including design overviews, walkthroughs, best practices, and eventually interesting case studies. I hope you will subscribe and participate with us in the conversation

ECM For the Masses

When speaking with customers about the content management features in SharePoint 2010, we often refer to the release as being about ECM for the Masses.  I wanted to take this first post to outline a little bit about our approach to designing and building SharePoint 2010, and how that has helped us deliver on that vision.

• Enterprise Ready – This is all about ensuring that SharePoint more easily scales to the amount of content the largest Enterprises deal with and delivers consistently high performance and reliability at any scale. In addition we provide the feature depth, customizability and extensibility that Enterprises need to support the full breadth of business scenarios around content.


• Easy to Use – Our focus here is on 2 audiences. First, the features must be Easy for the Information Worker, with best-in-class usability, providing supreme user acceptance and speeding deployment and adoption. Second, the product must also be Easy for IT, providing great functionality OOB that is fast to deploy and easy to manage at the Enterprise, Divisional, Team and Workgroup levels.


• Everyone Participates – This is all about ensuring that Everyone in the organization has access to and benefits from the functionality offered by the ECM features – not just a few specialists who have been specially trained, or for whom the organization can justify a high per seat price. This also means that the capabilities can be adjusted to suit the needs of everyone in the organization; from minimal interaction to highly structured and complex workflows – everyone sees exactly as much as they need.

It is my hope that, as we go through the features over the next several months that you will see the impact of those pillars on the product, and how they have helped us to deliver ECM for the masses.

Around two-and-a-half years ago (at the AIIM conference in Philadelphia in May of 2006), my counterparts at IBM and EMC and I started discussing the need to form a group to create an open services standard for interacting with Enterprise Content Management systems (like SharePoint, IBM FileNet P8, EMC Documentum, etc.) in a uniform way. An earlier blog post explained the full rationale – but in short many customers and partners made it apparent that having to create one-off “connectors” between each application (like eDiscovery applications, Portals or Business Process Management systems) and ECM system was making it hard for customers to use more than one ECM system and for partners to build great applications that could “just work” with whatever systems a customer is using.

And now, after working with many other vendors like Alfresco, Nuxeo, OpenText, Oracle, SAP, and others on the CMIS specification, forming a Technical Committee at OASIS to deliver that specification as a truly open standard, and having four “plug-fest” events where we’ve tested actual (prototype) implementations of the spec together to make sure it would work in the real-world – I’m thrilled to announce that on October 23, 2009 Version 1.0 of the CMIS specification entered OASIS’ public review process.

“Public Review” — What does that mean?

Those of you who aren’t familiar with the mechanics of the OASIS standards process (i.e. nearly everyone) are probably wondering what “public review” means, and how it relates to everyone’s ultimate goal of having a final 1.0 specification available so that everyone can start supporting CMIS in their applications. (If you want the full details of how the Public Review Process works, you can read OASIS official “Technical Committee Process” rules – but the summary below is a bit more self-contained and user-friendly).

Public Review is one of the final stages of the OASIS standardization process –it means that the members of the Technical Committee think the spec is (almost completely) done, and that we’re soliciting feedback from the general public about what changes (if any) they’d like to see in the
specification before it becomes the final CMIS 1.0 standard. The public review period lasts for 2 months – so it will continue until December 22, 2009.

Anyone can look at the spec and send us comments–I’ve also copied the full public review announcement below for reference.

The Technical Committee is then required to review and respond to all comments – which could include updating the specification, or responding to those comments without making a change (which we would likely do if the comment is asking for new features or big enough that it would be better deferred to a future version of the specification). If those comments result in substantive changes, then the updated spec would undergo a shorter (15-day) additional round of public review. If not, then the Technical Committee will submit the CMIS 1.0 specification (possibly with some minor clarification updates) for a final approval vote by the OASIS membership – a process which takes about a month.

Once the final OASIS approval vote is closed (assuming of course that CMIS gets sufficient votes to pass), CMIS 1.0 is (finally) an OASIS standard!

OK… so when will CMIS 1.0 be final?

Those of you keeping count from the above paragraph already figured this out – but CMIS is on track to become a final 1.0 standard sometime in the first 3 months of 2010 (exactly how soon will depend on the volume of comments we get in Public Review and the changes required to address them).

Given all of the work that the Technical Committee has done in writing & testing the spec so far, we aren’t expecting to make many (if any) substantive changes – but of course if there are any issues in the spec that will hamper it’s real-world adoptability we want to hear about and address those now in Public Review, rather than waiting until the 1.0 standard is final (when making changes will require a whole new version of the specification.) So please do review the spec and give us your feedback!

When can I expect vendors (including Microsoft) to start supporting CMIS in their products?

At this point, pretty much every vendor in the ECM space is really motivated to start supporting CMIS in their respective products. We’ve all seen the excitement from customers about CMIS — for example, a recent AIIM survey showed that 15% of organizations are already interested in using CMIS. (This is an unbelievable number for a standard that isn’t even final yet!)

Of course, the prerequisite for all this is a final, OASIS-ratified 1.0 standard. While several companies have released prototypes based on interim drafts (which are wonderful proof-points that CMIS is ready for real-world implementation), look for vendors to start disclosing specific plans once the specification is final.

For Microsoft’s part, we announced at the ARMA 2009 Conference and at our own SharePoint Conference in the last two weeks that we are planning to deliver support for CMIS within SharePoint 2010.

Since that announcement we’ve gotten lots of requests for additional details (“give me an exact date!”, “tell me exactly what functionality will be included”, etc.) I wish that this blog post could be the place to provide more detail – but it’s simply not possible at this time. Here’s why — those of you who attended the SharePoint Conference last week have seen that SharePoint 2010 is looking pretty shiny and polished. But until the CMIS 1.0 specification is final, we can’t realistically commit to exact dates when our CMIS support would be ready. This means that our plans need to be flexible to balance the following needs:

1. Not rushing the finalization of the CMIS 1.0 specification in a way that would compromise its quality


2. Release CMIS support as soon as possible for SharePoint 2010 that meets the interoperability needs of our customers and partners

We’re definitely looking forward to having the CMIS standardization process complete so we can lock-down our plans to the point where we can share additional details. Please stay tuned for more information.

Ethan Gur-esh,
Program Manager
CMIS Specification Editor
CMIS Technical Committee Secretary

Copy of the OASIS Public Review Announcement:

To OASIS members, Public Announce Lists:

The OASIS Content Management Interoperability Services (CMIS) TC has 
recently approved the following specification as a Committee Draft and 
approved the package for public review:

Content Management Interoperability Services (CMIS) Version 1.0

The public review starts today, 23 October 2009, and ends 22 December 
2009. This is an open invitation to comment. We strongly encourage 
feedback from potential users, developers and others, whether OASIS 
members or not, for the sake of improving the interoperability and 
quality of OASIS work. Please feel free to distribute this 
announcement within your organization and to other appropriate mail 
lists.

More non-normative information about the specification and the 
technical committee may be found at the public home page of the TC at:
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=cmis. 
Comments may be submitted to the TC by any person through the use of 
the OASIS TC Comment Facility which can be located via the button 
marked “Send A Comment” at the top of that page, or directly at:
http://www.oasis-open.org/committees/comments/index.php?wg_abbrev=cmis.

Submitted comments (for this work as well as other works of that TC) 
are publicly archived and can be viewed at:
http://lists.oasis-open.org/archives/cmis-comment/. All comments 
submitted to OASIS are subject to the OASIS Feedback License, which 
ensures that the feedback you provide carries the same obligations at 
least as the obligations of the TC members.

The specification document and related files are available here:

Editable Source:
http://docs.oasis-open.org/cmis/CMIS/v1.0/cd04/cmis-spec-v1.0.doc
PDF:
http://docs.oasis-open.org/cmis/CMIS/v1.0/cd04/cmis-spec-v1.0.pdf
HTML:
http://docs.oasis-open.org/cmis/CMIS/v1.0/cd04/cmis-spec-v1.0.html

Schema:
http://docs.oasis-open.org/cmis/CMIS/v1.0/cd04/CMIS-Core.xsd
http://docs.oasis-open.org/cmis/CMIS/v1.0/cd04/CMIS-Messaging.xsd
http://docs.oasis-open.org/cmis/CMIS/v1.0/cd04/CMIS-RestAtom.xsd


OASIS and the CMIS TC welcome your comments.


Mary P McRae
Director, Technical Committee Administration
OASIS: Advancing open standards for the information society
email: mary.mcrae@oasis-open.org
web: www.oasis-open.org
twitter: fiberartisan #oasisopen
phone: 1.603.232.9090