Experiences with Binary and DNA abstractions

Since over a year now, I have been working on an Innovation product on SharePoint 2010 (read more…) and I would like to share some my experiences, architecting and developing it. Below I have tried to group my experiences into technical and non-technical headings and also shared some of the interesting resources that I went through.

• Business Context
• Understanding the business problem.  (Business Drives)
• Why SharePoint?
• Defining Common Goals.
• Understanding your Customer’s Customer.
• Architecture & Design
• SharePoint Artifacts  (SharePoint 2010 Architectures Overview)
• Storing data in Lists vs. SQL Tables (SharePoint Large List Performance: SPMetal vs. SPQuery)
• SharePoint Foundation vs. Standard vs. Enterprise specific Features
• Site Definitions/ Site Templates / Master Pages / Page Layouts …
• Adventures with Variations
• SharePoint 2010 Cache (read more…)
• Search
• Loose coupling  (Dependency Inversion Principle)
• Avoiding “Pattern Pathology”
• Performance
• Tips & Tricks while using Entity Framework (EF Deep Dive)
• Embracing JavaScript via jQuery (Douglas Crockford, John Resig, Paul Irish)
• Versioning and Upgrades
• Data Import
• Unit & Integration Tests (Unit Testing Best Practices, using MSDTC for Integration Tests)
• Team Lead
• Communication is the ‘key’
• Vertical vs. Horizontal slicing (Code Leaders and Beautiful Teams)
• It is hard to Unlearn
• Process
• Adopting Team Foundation Server
• Sprint Planning
• User Stories / Tasks / Bugs
• Daily Standups
• Daily Builds

In the next few posts I would try to expand some of these headings and talk about them in more details…

First hands on Roslyn . . .

 

 

 

 

Microsoft “Roslyn” CTP

Traditionally, compilers are black boxes – source code goes in one end and object files or assemblies come out the other end. The Roslyn project changes that model by opening up the Visual Basic and C# compilers as APIs. These APIs allow tools and end-users to share in the wealth of information the compilers have about code. The Roslyn CTP previews the next generation of language object models for code generation, analysis, and refactoring, and the upcoming support for scripting and interactive use of VB and C#.

http://msdn.microsoft.com/en-us/roslyn

HTML5 Diaries – Canvas & SVG

How to choose between Canvas and SVG ?

Fundamentally, Canvas is a pixel-based, immediate mode graphics system that works best with a smaller surface or a larger number of objects.
While, SVG is a shape-based, retained mode graphics system that works better with a larger surface or a smaller number of objects.

For more information, visit :

http://msdn.microsoft.com/en-us/library/gg589488(v=vs.85).aspx
http://dev.opera.com/articles/view/svg-or-canvas-choosing-between-the-two/
https://developer.mozilla.org/en/Canvas_tutorial
http://www.canvasdemos.com/

HTML5 Diaries - Semantic Markup

<div> is old fashioned now . . .

Food for Enthusiasts:

http://dev.w3.org/html5/spec/Overview.html
http://www.html5rocks.com/en/
http://channel9.msdn.com/Events/Build/Build2011?sort=sequential&direction=desc&term=&t=html5%2Band%2Bjavascript

WebSockets + Long Polling + SignalR === Awesome;

Steps for creating a chat client using ASP.net 4.5 :

1) Install Visual Studio 2011 Developer Preview (download)
2) Install NuGet Package Manager (download)
3) File -> New -> Project

4) Right Click on References -> Manage NuGet Packages

1) Search “SignalR” and install the package

2) Add 2 files “Default.aspx” and “Chat.cs” and change the code as below:

(*Note : I am referring to my C# type and methods in Javascript )
3) Done.

You may think this is trivial but believe me, it isn’t ..

This is the most exciting piece of technology that I have seen so far from BUILD (offcourse after Windows Native Apps in HTML5 & Javascript) so I would encourage everyone to start playing with VS 2011.
And if you haven’t seen yet what happened in BUILD than I would strongly recommend you see it here.

Web development is taking a whole new perspective with HTML 5 and Javascript . . . .

Code Leaders and Beautiful Teams . . . . .

 

I came across this nice presentation done by Roy Osherove at NDC 2010 where he talks about team types and some very good fundamental aspects of directing the team as a technical leader.

Here is the torrent link for all the videos - NDC2010 Sessions Torrent . I especially liked his Song at the end of the presentation (below) 

Here are some really good excerpts from the original presentation :

Diving & Exploration

Some very good SharePoint links that I came across in last few weeks :

• SharePoint 2010 Architectures Overview on MSDN
  A pretty decent article covering the breath of SharePoint 2010. Although I don’t like when it says : “The Silverlight client object model in SharePoint is not supported on Windows Phone 7.”
  
• Application Lifecycle Management in Microsoft SharePoint 2010
  Good article to learn what things should be considered when setting up team development environments, establishing upgrade management processes, and creating a standard development model for SharePoint projects.
  
• SharePoint Server 2010 performance and capacity test results and recommendations
  A set of 9 white papers describing the performance and capacity impact of feature like BCS, Large Lists, My Sites, Search etc. included in Microsoft SharePoint Server 2010.
  
• SharePoint 2010 Social Networking
  Excellent series on understanding the details of Social Computing features in SharePoint 2010. I just love this Social Computing processes diagram explaining all the things that done behind the scenes.

Enjoy Learning !!

First touch to Windows Phone 7

 

Visual Studio 2010 Power Tools

Here are few features I liked from the VS 2010 Power Tools released last week :

Visual Studio 2010 Pro Power Tools

• Close button in tab as well
• Pinned Tabs
• Searchable Add Reference Dialog
• Triple Click to select a line
• Align Assignments (Ctrl + Alt + ] )

Visual Studio 2010 SharePoint Power Tools

• Sandbox Visual Webpart

(Although there are few limitations which are mentioned in the tool description tab here)

Diving & Exploration

Here are some gems I found last month and half:

LINQ

• DevDays 2010 - Tips, Tricks and Optimizations by Scott Allen
• DevDays 2010 - Beyond Queries by Scott Allen

SharePoint 2010

• Hard Limits / Configurable Limits / Soft Limits of SharePoint 2010 by Sahil Malik
• Technical diagrams (SharePoint Server 2010)
• Setting Custom Error Page in SharePoint 2010
• Deployment Overview (SharePoint 2010)

Zion Rebuild . . . . . . . . Again !!!

Right from the RFP meetings to the daily scrums and the final PROD release calls/emails, it has been a tremendously exciting journey once again with this project. I am immensely happy to announce the launch of our new SharePoint site: http://sharepoint.microsoft.com

The entire team learnt & applied a ton of exciting stuff in SharePoint 2010 including :

• LIVE ID Integration with SharePoint 2010
• 2010 Media Player Integration
• Localization using Variations + lots of Hack
• [PageQueryString] in Content Query Webparts
• SharePoint Tools in Visual Studio 2010
• Caching API’s for accessing SharePoint List Data
• Bing Maps Geo-code service
• HTML Compliance / Accessibility standards
• jQuery

SharePoint 2010 is a huge successor of 2007 and since last 1 year I have been enjoying more and more working on this product . . . . . .

Diving & Exploration

After a long time, here are some gems i found in last couple of months :

SharePoint 2010

• MSDN Client Object Model - Retrieve List Items
• MSDN - Deployment overview (SharePoint Server 2010)
• CodePlex - SharePoint Developer Tools for Visual Studio 2010 (including Visual Webpart for Sandbox)
• SharePoint’s Sasquatch Memory Leak
• Certification exam details for SharePoint 2010 published

Azure

• Azure’s Service Bus and EnergyNet (PDC Day 0)

General

• J.D. Meier’s - Mapping Out the Microsoft Application Platform at a Glance
• 53 CSS-Techniques You Couldn’t Live Without
• Three Dimensions of a Software Programmer: How to get things done

First Look at Windows Phone Developer Tools (CTP)

I just downloaded the Windows Phone Developer Tools and tried the Code Samples on my Windows 7 PDC Laptop. Here are few snapshots of the emulator:

If you want to get your hands more dirty than here is the Windows Phone 7 Series Developer Training Kit

First Look at IE9 – GPU-powered HTML5 graphics

IE9 Developer Platform Preview : http://ie.microsoft.com/testdrive/

 

Border Radius

T-Shirt Designer

SVG-oids

Business Charts

Atlas

SharePoint 2010 Branding

Just saw these interesting Tips from Randy Drisgrill on branding SharePoint 2010 sites and also a Starter Master Page for SharePoint 2010, with all the detailed comments to understand each and every section of the 2010 Master page. . . . .

Diving & Exploration

I was busy with PDC so this post is coming a bit late. Here are some of the gems that I had explored lastly:

• Developing SharePoint Applications : A series of articles from the patterns and practices developer center on building Collaborative Applications using SharePoint. One of my college Todd Baginski has also contributed to these articles
• SharePoint 2010 Sandbox Solutions : Yet again excellent article series from Sahil Malik explaining all the details that you would possibly need to develop Sandbox Solutions in SharePoint 2010.
• ASP.net Performance and Microsoft AJAX CDN.
• ASP.net 4.0 and Visual Studio 2010 Web Development Beta 2 Overview:  Whitepaper explaining the new changes coming in ASP.net in .Net framework 4.0.
• What’s new in WCF 4.0 : 3 Part series explaining some of the new features of WCF 4.0.
• Visual Studio 2010 and .NET Framework 4 Training Course: Yet another **FREE** training course form channel9 with lots of videos and hands-on-lab.

Using the Built-in Silverlight Media Player of SharePoint 2010

This summary is not available. Please click here to view the post.

Dialog Platform in SharePoint 2010 & How to open the Edit Form Dialog for List Item

Dialog Platform in SharePoint 2010:

One of the New User Interface Platforms in SharePoint 2010 is ‘The Dialog Platform’
A dialog is essentially a <div> which gets visible on demand and renders the HTML using a background overlay creating a modal dialog like user experience.

We can show an existing div from within the page or a different page using a URL inside the dialogs.
When we pass the URL to the dialog it looks for the Querystring parameter “IsDlg=1”. If this parameters exists than it would dynamically load the "/_layouts/styles/dlgframe.css” file. This file overrides the “s4-notdlg” class items as “display:none”, which means that all items with this class would not get displayed in Dialog Mode. 
So if we go to the v4.master page we can see that this class is used by the Ribbon control to hide the ribbon when in dialog mode:

 

How to open the Edit Form Dialog for List Item:

In SharePoint 2010 The URL for opening the Edit Form of any list item looks like something like this :

”http://intranet.contoso.com/<SiteName>/Lists/<ListName>/EditForm.aspx?ID=1&IsDlg=1”

ID is the list item row identifier and as discussed above the IsDlg is for the dialog mode.

Now to open a dialog we need to use the SP.UI.ModalDialog.showModalDialog method from the ECMAScript Client Object model and pass in the url of the page, width & height of the dialog and also a callback function in case we want some code to run after the dialog is closed.

<script type="text/javascript">
         //Handle the DialogCallback callback 
              function DialogCallback(dialogResult, returnValue){ 
              } 
  
          //Open the Dialog 
          function OpenEditDialog(id){ 
           var options = { 
url:&quot;http://intranet.contoso.com/<SiteName>/Lists/<ListName>/EditForm.aspx?ID=&quot; + id + &quot;&amp;IsDlg=1&quot;, 
             width: 700, 
             height: 700, 
             dialogReturnValueCallback: DialogCallback 
             }; 
            SP.UI.ModalDialog.showModalDialog(options); 
          }
</script>

The .js files for the ECMAScript Object Model (SP.js, SP.Core.js, SP.Ribbon.js, and SP.Runtime.js ) are installed in the %ProgramFiles%\Common Files\Microsoft Shared\web server extensions\14\TEMPLATE\LAYOUTS directory.


Here is a good MSDN link explaining the Client Object Model Distribution and Deployment options available in SharePoint 2010.

Summary of all PDC09 – Announcements / Product Features / Cool Stuff

 

Silverlight

• Silverlight 4.0 Features & More:
  • Support for Webcam, Microphones, Right Click, Mouse Wheel
  • New Rich Text control
  • Clipboard Access
  • Print Preview Dialogs
  • Authorization support in Client HTTP Stack (NTLM, Basic, Digest)
  • Running outside sandbox for Trusted Out of browser applications
    • HTML rendering control inside Silverlight
    • Local file system access
    • Keyboard in full screen
    • COM automation support (read word/excel files)
    • Notification Popups
• Screen Presentation Framework (MVVM + Prism) for Silverlight
• Silverlight Media Framework - Smooth Streaming Porting Kit
• Microsoft XAML Toolkit : Provides Static Analysis, Localization, & XamlDom for .NET/Silverlight XAML
• ADO.NET Data Services is now “OData”  - Open Data Protocol & WCF Data Services 
• .NET 4 Location API - Installed by default with .NET 4 and supported in Visual Studio 2010. (System.Device.dll) See WHDC site for Sensors/Location : www.microsoft.com/WHDC/Sensors
• Bing Maps Silverlight Control
• Advanced Performance & Profiling in Silverlight 4
• WCF RIA Services for Silverlight

SharePoint

• SharePoint 2010 Features :  (*FREE Channel9 SharePoint2010 Developer Course)
  • SharePoint 2010 Designer New Tools
    • BDC Entities
    • List Schema
    • Custom Actions
    • Task/Approval Designer
  • Visual Studio 2010 SharePoint Tools
    • Web Part, BCS and Workflow designers
    • View SharePoint site in Server Explorer
    • Support for SharePoint Sandboxed Solutions
    • Import WSP from SharePoint Designer
    • Packaging Explorer
  • Platform
    • Web UI Framework
    • Silverlight
    • XSLT Views
    • LINQ
    • REST
    • Client OM
    • External Lists
    • Improved Event Model
    • Improved Solutions Lifecycle
• Word Automation Services in SharePoint 2010 + Open XML SDK
• “Identity normalization” in SharePoint 2010 – Services in Single Farm, Windows Identity Foundation Identity Delegation Feature
• Identity and Web2.0 Services – OAuth Token Authentication mechanism in SharePoint 2010

Windows Azure

• Windows Azure + Windows Identity Foundation + ADFS2 : https://federatedidentity.cloudapp.net/
• Windows Azure Platform AppFabric - provides secure connectivity as a service to help developers bridge cloud, on-premises, and hosted deployments
• Velocity is now “Windows Server AppFabric Caching”
• Dublin is now  “Windows Server AppFabric” - The Windows Server AppFabric supports WCF and Workflow services by:
  • Providing runtime databases
  • Capabilities for monitoring, persistence, hosting and caching
  • Tooling and scripting to manage and monitor WCF and WF services
• Windows Azure :
  • Features
    • Blobs – Provide a simple interface for storing named files along with metadata for the file
    • New Type - Page Blob
    • New Operations – Updated, Copy, Snapshot, Lease Blob
    • New Ways of Accessing and Service Blob Content -
      • Custom Domain Names
      • Root Blob Container
      • Content Delivery Network access
      • Shared Access Signatures
    • Entity Group Transactions - Atomically perform multiple insert/update/delete
      over entities in same partition in a single transaction
    • Dequeue count returned with message
  • Futures
    • Administrator privileges in the VM
    • Deployment of pre-configured VM images
    • User-selectable geo-locations for replicas
    • Secondary indices on tables
    • New storage option “Drives” - Provides durable NTFS volumes for Windows Azure applications to use (Feb 2010)
    • Remote terminal server access to VMs
    • Microsoft Sync Framework and ADO.NET Sync Services - Tools for data synchronization with SQL Azure
• TCO and ROI Calculator - provides a customized estimate of potential cost savings you (or your company or organization) may achieve by building on the Windows Azure Platform
• Windows Azure Diagnostics - SDK component providing distributed monitoring & data collection for cloud apps
• Visual Studio 2010 RTM and SQL Azure Futures:
  • Connect to SQL Azure – Data | Add New Data Source (DataSet & EDM)
  • EDM Conceptual Data Modeling Designer – Generate database
  • Server Explorer – Browse
  • Data Sources Window (DataSet, EDM, LinqToSQL, data binding tools)
  • Configure SQL Data Source
• File Classification Infrastructure (FCI) - provides many avenues to be part of end-to-end data lifecycle management solutions
• “Houston” - Project Houston will provide a easy to use cloud based database development experience
• Access Control Service - claims based access control for REST web services
  • Implements OAuth Web Resource Authorization Protocol (WRAP) & Simple Web Tokens (SWT)
  • Integrates with AD FS v2

Surface & Windows 7

• Multitouch on Windows 7
  • Manipulation Events
  • ScrollViewer Panning Support
  • WPF Controls Styles with Panning Enabled
  • Multi-Touch on the web with Silverlight
    • Manipulation and Inertia Process API
• Windows API Code Pack – Managed Class Library to access to Windows (7) features
• “Surface Toolkit for Windows Touch” – Would be coming shortly after the WPF4 Launch
• Surface Futures:
  • Cheaper
  • Thinner
  • Vertical
• Surface SDK & Resources now available free

Visual Studio, WCF, WPF

• WCF 4.0 Features:
  • Default Endpoints, Protocol Mapping, Binding Configurations,Behavior Configurations
  • File-less Activation (no .svc)
  • Simple Ad-hoc Service Discovery
  • Service Announcements
  • Routing Service
  • Message Filters and Content-based Routing
  • Automatic Help Page
  • HTTP Caching Support
• WPF 4.0 New Features: Here is an excellent blog series from Lester covering all the below features with sample code:
  • MultiTouch
  • DatePicker\Calendar
  • Datagrid
  • Visual State Manager (VSM)
  • Easing Functions
  • JumpLists
  • Pixel Shader 3 support
  • Cached Composition
  • Full trust Xbap Deployment
  • Binding to Dynamic Objects
  • Custom Speller Support
  • Script Interop in Xbap
  • UIA Virtualization
  • UIA SynchronizedInputPattern
  • Input Key\Gesture Binding
  • TextFormattingMode
  • CaretBrush
  • SelectionBrush
  • Bindable Run
  • ClearTypeHint
  • LayoutRounding
  • Updated File Dialogs
• Manycore & .NET 4.0 with Visual Studio 2010 (Concurrency):
  • ThreadPool.QueueUserWorkItem
  • System.Threading.Tasks.Task + System.Collections.Concurrent
  • New Sync Primitives in .NET 4
  • Visual Studio 2010 Concurrency Profiler Views + Parallel Debugger Toolwindow
• Visual Studio 2010 Extensibility – Extensions Gallery & VSX Samples

ASP.net & SQL Server

• ASP.net 4.0 Futures:
  • OOB Helpers for Image manipulation, Email (and signup email with verification), Background tasks, File upload with progress updates
  • Support for HTML5 - <video>, <audio>, <menu> tags, Web Storage
  • AdoNetDataContext – to get data from Windows Communication Foundation web services (*.svc)
  • IntermediateDataContext to cache data in Web Storage
  • Velocity – distributed cache platform
  • Active Record design pattern to keep the database schema in sync with the Object-Relational Mapper.
• FREE ASP.NET 4 & MS Deploy Hosting
• Microsoft SQL Server 2008 R2 StreamInsight
• SQL Server 2008 R2 Embedded BI: Visual Studio 2010 Report Viewer Controls
• SQL Server Modeling Services
• Statistics Used by the Query Optimizer in Microsoft SQL Server 2008

Microsoft Research & Labs

• Infer.NET - framework for running Bayesian inference in graphical models (from MSR)
• “Axum” : A .NET Language for Safe and Scalable Concurrency
• “Holmes” - Automated Statistical Debugging using Path Profiles (from MSR)
• “Darwin” - An Approach for Debugging Software Regressions (from MSR)
• “Project Trident” - A Scientific Workflow Workbench (from MSR)
• “Pivot” by Live Labs : Visualizing collections of objects, Revealing hidden patterns within data, Acting on discovered insights
• DevLabs: Reactive Extensions for .NET (Rx)
• Trustworthy Data Collaboration SQL Azure Codename ‘Vidalia’
• Microsoft Codename “Dallas” - Content Brokerage and Discovery platform

Other

• Messenger Web Toolkit
• Concurrency Runtime (ConcRT) & User Mode Scheduling (UMS) for C++
• “Snoop” - A random utility made to simplify visual debugging of WPF applications at runtime.
• “VMmap” - Process virtual and physical memory analysis utility from sysinternals
• DryadLINQ - Declarative programming style of LINQ for clusters
• Windows Performance Tools (WPT) Kit - contains performance analysis tools
• Windows Ribbon Framework Developer Guides
• .NET Micro Framework 4.0 Porting Kit and .NET Micro Framework 4.0 SDK
• Image Builder Wizard (IBW) – Quick and Easy Embedded OS Creation
• Microsoft adPlatform for Developers
• Windows Identity Foundation - Extension to .Net for claims-based identity
• Software Certification Toolkit – For ISV Application Readiness and Certification
• UC 14 Metro - OCS 14/OC 14 platform early adopter program

@PDC09 – Day 3 – Developing custom applications in SharePoint 2010 using ASP.net, WCF, REST

SharePoint Web Services

• REST
• Client Object Model
• ASP.net
• WCF

REST

• Access to sharepoint lists and library data ( …/_vti_bin/listdata.svc)
• Lists and List items map to resources (listdata.svc/Employees(123) )
• Operation are mapped with HTTP verbs (GET, POST . . )
• Based on ADO.net Data Services
  • Flexible URL conventions for querying data
    • listdata.svc/Employees?$filder=JobTitle eq ‘SDE’
    • $orderby=Fullname
  • Multiple representations
    • JSON and Atom (full AtomPub support)
  • Batching, ETags, Paging, Documents

Demo . . .
REST query is converted to CAML query…..

Integrated Development Experience in VS 2010 and VS 2008 SP1 using just “Add Reference”

• Data Source configuration has a new item to connect to SharePoint
• Master Details type application development

Client Object Model

• Supports
  • Site Operations
  • Lists, ListItems
  • content types, fields
  • but not administration namespace, farm objects
• Implemented as WCF Service
  • …/vti_bin/client.svc
  • It passes batch command to the WCF service (in XML)
  • than the server executes and returns the data back to client (in JSON Other SharePoint Web Services
• Advanced Operations
  • User Profiles
  • People
  • Search
  • Enterprise Metadata
  • Document Coauthoring
  • Excek REST web services
  • Publishing
  • Workflow
  • BCS web services
  • ….. see MSDN
• we can also extend sharepoint and create custom web services
  • ASP.net
  • WCF (is now recommended for sharepoint 2010 custom service development)

WCF Services

• Problem
  • IT Admin configures IIS
    • Different authentication schemes can be configured (Basic, Claims, Windows Integrated . . .)
    • Multiple IIS address bindings
  • Developer configures WCF Service endpoint
• Solution
  • Dynamic (Programmatic) configuration
    this would be in RTM only
  • SOAP
    • Specify Service Factory in SVC file
    • Add MEX endpoint behavior attribute to the service class
    • Deploy SVC service to ISAPI Folder
  • REST

Demo of creating custom WCF service inside sharepoint

• sandbox solutions doesn’t support services

WCF Service Configuration

• Service.svc
• SharePoint Service Factory creates the endpoint based on all authentications
  • service.svc/negotiate
  • service.svc/ntlm
• SharePoint Http Modules