PowerPivotGeek

What’s your favorite PowerPivot for SharePoint feature?

powerpivotgeek — Sun, 07 Mar 2010 08:27:18 +0000

Ok. Now is is time for you guys to speak up! Here is a little informal poll. What is your favorite PowerPivot feature? I’ll tell you mine and then I’ll let everyone chime in via the post comments. Join in the fun!

My favorite PowerPivot feature isn’t a feature of PowerPivot at all. I love the fact that since we have this ‘document-centric’ view of the data – all of the PowerPivot data is embedded in the workbook itself. This means that all of the build-in SharePoint features automatically apply to PowerPivot as well. Here is a quick list of what comes to mind to me:

Check in/out. Users can have an collaborative interaction with the data by using check in/out on the embedded workbook file itself.
An approval workflow. After a check in, or a new posting of the workbook, you can kick off an approval workflow – i.e. you can route the approval around a series of users until the data shows up in the document library.
Document library custom modifications. With SharePoint you can add your own properties to the file and include that in the document library along with everything else. SharePoint becomes a development platform with the PowerPivot workbooks as the basis for the custom changes. All of this is created and managed by end-users themselves.
Automatic retention policies. Since the data is just part of the document, you can have business rules that automatically delete the data. For example, you might have a rule like: “items in this document library are automatically deleted after 180 days” (very appropriate for financial data).
Many of the “data management” activities are not part of the PowerPivot application – but rather end-users do it themselves. If they want to reorganize how the data is stored on their site, then they just copy PowerPivot workbooks to a new place. They don’t have to call up IT – they just do it! If the data no longer applies to their system, they just delete it. The data is the file – the file is the data.
Wastebasket. SharePoint has a built-in wastebasket feature. If you do delete a file, it isn’t really gone. Until the administrator empties the wastebasket, the data still lives on – and the user can copy it back to their document libraries themselves. Again, since the data is the file – you don’t need any special PowerPivot capabilities to do this.
Security is all end-user driven. There is no special security settings for PowerPivot. Since the data is the file, we just use a user’s access rights to the file to determine their access rights to the embedded data. If an end-user drops a user’s read permissions, then they cannot access the embedded data. It is that simple. There are no special utilities or settings to configure. Users just use their regular SharePoint capabilities on the PowerPivot workbook.

I could go on and on – but you get the idea. SharePoint adds tremendous value in the PowerPivot world.

So that is my favorite feature, what’s yours? Add a comment and join in!

A Peek Inside: Unloading PowerPivot data

powerpivotgeek — Fri, 05 Mar 2010 05:58:18 +0000

Ok folks. Here comes a cool one.

As I am sure that everyone is aware by now, one of the most powerful new features of PowerPivot is that we do on-demand loading of embedded data. Well, if you are going to load it, then you had better at least think about unload it at some time. There is probably some law of thermodynamics that applies here.

First rule for deleting data is that we don’t touch data if there are users currently connected to it. But other than that, any databases loaded or cached by the PSS (i.e. their database folder is contained in the SSAS Backup folder tree) are candidates for being unloaded.

There are two ways that databases can be unloaded in PowerPivot:

a) Inactivity

If a database has not been accessed for a long time, then it is a good idea to start to free up resources for it. However being too aggressive means that you may have to re-read content which places a larger load on SharePoint. One of a farm’s most critical resources is the amount of traffic that has to be moved to and from the content database. So we want to minimize its load as much as possible. This is particularly true if the content is large – which is likely for PowerPivot workbooks. We have designed PowerPivot so that inactivity checking is a two stage algorithm. Every 30 minutes the PowerPivot System Service (PSS) scans the databases in the instance map assigned to the server’s service application looking for:

A loaded database with its ‘last accessed’ timestamp longer than 48 hours (2 days). If it finds one, the PSS detaches the database from the SSAS instance. The database remains in the cache (a collection of detached databases on a server) so it is available for quickly coming back if a user accesses the database. If this happens, all the PSS has to do is to issue an Attach XMLA command – it does not have to go back to the SharePoint content database and re-read the workbook.
A cached database with its ‘last accessed’ timestamp longer than 120 hours (5 days). If it finds one, the PSS deletes the database from the cache by deleting the database folder (and all its files) from the hard disk. If a user accesses the database after it was deleted from the cache then the data has to be re-read from the workbook (which is stored in the SharePoint content database).

b) New Content has arrived

If a file is updated in SharePoint (whether or not file versioning is enabled or not), then the any databases loaded or cached for that file can be unloaded from memory and deleted from the server cache immediately. Ordinarily you would think that it would be over aggressive to delete all old databases.

Every 30 minutes the PSS scans the databases in the instance map assigned to the server’s service application looking for content that is newer than the content loaded or cached. The instance map contains the SPFileVersion.Created property of the file version that is loaded. If the ‘latest version’ of the file has a SPFileVersion.Created that is later than the instance map then we know that new content has arrived. If the database is loaded, then we delete the database from the SSAS instance by sending an XMLA Delete command. If the database is cached, then the PSS deletes the database from the cache by deleting the database folder (and all its files) from the hard disk.

Note that we cannot use the SharePoint versioning system to determine if a version has changed. We have to use this timestamp method because the SharePoint versioning system can be disabled. In fact it is disabled by default. When it is disabled all files look like version 1.0. We are able to use the file version “Created” property because files in SharePoint are read-only. There is no in-place editing. If you edit a file, then behind the scenes you are creating a new version and it has the “Created” timestamp of the save operation.

—————————-

Being a geek, you are probably asking if you can change any of these thresholds or behaviors. The answer is “Sorry, No. Maybe next version.”

Enjoy!

PowerPivot and Windows 7

powerpivotgeek — Fri, 05 Mar 2010 04:37:46 +0000

Recently I had a conversation with one of our excellent beta testers and I want to make sure that everyone was clear about running PowerPivot on Windows 7.

Windows 7 is only supported for PowerPivot for Excel; Windows 7 is not a supported OS for PowerPivot for SharePoint

Several months ago we purposely made the decision to not support Windows 7 on the server. If you do try it, then good for you (I like adventures too) but you are totally on your own. Please don’t post any server-side question to the forums and other support channels. We cannot be asking “oh, is this running under Windows 7” with every bug report. We don’t support Windows 7; we haven’t tested under it; and we don’t even do adhoc “oh, I wonder what will happen here”. We don’t know where or if Windows 7 breaks anything. The SharePoint story with Windows 7 is pretty clear. They provide guidance on a “standalone”, non-production SharePoint installation. The problem is that PowerPivot requires a “Farm” and “Complete” installation of SharePoint Server. So we don’t believe that we have adequate test coverage to support an edge configuration for SharePoint. PowerPivot for SharePoint requires a server OS, i.e. either Win2K8 SP2 or Win2K8 R2.

For the client, we have a totally different story. Windows 7 is a supported (and the preferred) platform for Office, Excel and PowerPivot for Excel.

I appreciate that this is an issue with some folks. And I understand where you are coming from, but we cannot help it. Windows 7 is an unsupported platform for the PowerPivot server components.

Obviously if you have gone down this path, we’d love to hear about it. If you have the spare bandwidth, feel free to play, and report back – but we cannot support you.

Clear? Enough said. (And now we return you to your regularly scheduled programming)

Thanks.

Installing the right version of SharePoint

powerpivotgeek — Fri, 05 Mar 2010 00:03:51 +0000

So you think that you have done everything right, but still SQL setup (in PowerPivot SharePoint integration mode) still reports “You don’t have a ‘Complete’ SharePoint installation.” How can this happen?

You might not have followed the installation guide. It tells you to first select “Farm” and then “Complete” on your first two screens after you enter your SharePoint product key. If you are installing on a domain controller, then the only selection that is supported is “Farm” and “Complete”, thus on a domain controller you won’t even be asked – it just defaults to this configuration. They don’t even offer you the possibility of running in an unsupported configuration. Just double check what you asked for. BTW: If you are running a RC0 version of SharePoint there is a known bug that a “Standalone” rather than “Farm” install is done. This is a RC0 only bug and it was fixed in a later build.
You might have installed the wrong version of SharePoint. You might have installed SharePoint Foundation Server instead of SharePoint Server. It turns out that this is very easy to do. It also does not ask you about “Farm” or “Complete” – it automatically installs a standalone SharePoint installation. So, how can you tell the difference (besides one returns an error; and one doesn’t) Here is a trick:

This is what the installation folder looks like with SharePoint Foundation Server:

This will generate a “standalone” installation – and PowerPivot cannot use it. While there is a setup rule about the version of SharePoint installed, under certain circumstances, instead SQL Setup reports that a SharePoint “Complete” installation was not done. Even the error message is a bit misleading since what the rule is actually checking it to see if the registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\14.0\WSS
Name: ServerRole = SINGLESERVER | APPLICATION

contains “APPLICATION”. So how can you check for a SharePoint Server 2010 distribution? Just look for some extra folders!

Here is what a SharePoint Server 2010 distribution looks like:

Notice the differences in the folders. There are additional folders for Excel Services, Access Services, and all of the new SharePoint 2010 services. Even though you might not actually turn on these features on your SharePoint site, the fact that the folders exist on the distribution tells you that you have the correct version of the product. This is the distribution you must be using for a good PowerPivot installation. Don’t worry about if it is Standard Edition or Enterprise Edition. In SharePoint, that is a difference in the key you install into the server. The bits are exactly the same.

Enjoy.

Keeping a workbook ‘pinned’ in memory

powerpivotgeek — Tue, 02 Mar 2010 20:51:15 +0000

One of the special considerations that arise when demonstrating PowerPivot is that you would like to ensure the workbooks that you are showing are kept in memory all of the time. PowerPivot does not have a built-in ‘pin’ feature to do this, but there is nice trick that you can use. Here is what I do:

Turn on SQL Agent.
The SQL Agent service account needs access to the content in SharePoint, i.e. the workbooks being demo’ed
Create a job with a schedule that runs every n minutes (I use every 5 minutes) and a single “SQL Server Analysis Services Query” step with the following characteristics:
1. Set the “Server” to be the URL to the workbook you are trying to keep loaded
2. Enter a command that you know will fail, or if you know one that would be OK, I typically use “SELECT * FROM SANDBOX” (which will fail) – we don’t need the query to run – all we need is a good connection and the embedded data will be loaded.

A valid question might be: “So tell me, Dave, why every 5 minutes? That seems way too often. After all this is a demo system and there isn’t memory pressure, so why running it so often??” Yes, that is true. The other reason for why the workbook data would not be loaded is the inactivity timeout @48 hours for detaching. But that also does not seem to apply. The reason why I use 5 minutes is data refresh. Since I typically demonstrate setting and running a data refresh, this means that new content is always being generated. When new content arrives in SharePoint, the PowerPivot cleaner thread checks every 10 minutes and it will delete unused old databases immediately. Thus you would have the ‘first access’ loading the new content – and with it, the built-in delay would happen. Running the job every 5 minutes means a forced load of the new data quickly. Your mileage may vary.

A new Diagnostic Guide posted in the Troubleshooting section

powerpivotgeek — Thu, 25 Feb 2010 02:17:06 +0000

I just posted a new diagnostic guide in the troubleshooting section:

“Diagnostic Guide for Usage and the PowerPivot Management Dashboard” by Ankur Goyal
http://powerpivotgeek.com/troubleshooting/diagnostic-guide-for-usage-and-the-powerpivot-management-dashboard/

Enjoy!

_-_-_ Dave

Uninstalling PowerPivot for Excel

powerpivotgeek — Wed, 24 Feb 2010 21:03:08 +0000

A trick for the client-side folks doing build-to-build reinstallations (or if you wish to clear all of the PowerPivot code from your client): besides uninstalling “Microsoft PowerPivot for Excel” (via control panel Add/Remove/Uninstall Programs, the exact wording varies depending on the OS you are running), you may also have to delete files in:

C:\Program Files\Microsoft Analysis Services, and

C:\Program Files(x86)\Microsoft Analysis Services (if running on an x64 machine)

This is because the Analysis Services OLE DB provider (and its components) are also installed in these folders. They are not removed during uninstall because we don’t know if there are other components that are using them. Thus you must be careful, particularly if you know that there are other dependent products that are using them before you delete the folders. Dependent products should be uninstalled before you delete these files.

When is null not a null?

powerpivotgeek — Mon, 15 Feb 2010 20:34:57 +0000

Recently there have been some discussions on the forums and in other areas concerning how PowerPivot handles nulls. As this has changed between CTP3 (back in November) and the upcoming RTM of the product, I thought that it would be a good posting. Here is the situation: suppose you have a simple calculated measure:

m1 = SUM(c1) / SUM(c2)

If c2 is NULL, then then you get #NUM as a result. No worries, pretty much expected. However, it is difficult to trap when this is happening. For example, with CTP3:

m1 = IF (SUM(c2) = 0, 0, SUM(c1) / SUM(c2))

You can use IFERROR and ISERROR but they are slower and have more overhead and should be avoided if possible:

m1 = IFERROR(( SUM(c1) / SUM(c2) ), 0)

-or-

m1 = IF(ISERROR( SUM(c1) / SUM(c2) ), 0, ( SUM(c1) / SUM(c2) ))

. . . however, regardless, m1 keeps returning #NUM, not 0. In CTP3 PowerPivot does not treat infinity as an error despite #NUM being displayed in Excel. This is effectively what is going on in this example, as NULL is being converted to zero (division by zero then yields infinity). The RTM version of PowerPivot will treat such a scenario as an error and IF/ IFERROR/ ISERROR will behave as expected.

(My thanks to Marius Dumitru, Russell Christopher and Brandon Unger for providing both the examples above and the answer.)

Enjoy.

_-_-_ Dave

Cannot add the specified assembly to the global assembly cache

powerpivotwahoo — Fri, 12 Feb 2010 19:09:33 +0000

This is part of my continuing series on debugging SharePoint and PowerPivot. In this case, I do not expect any PowerPivot users to ever encounter this error in the context of PowerPivot. However, the general SharePoint developer who is in the early stages of building and testing his custom solutions might hit this and since so many people on our team have been confused by it, I thought I would throw out some help for other developers. This error is hit when trying to deploy a SharePoint solution package to the farm. It could be an ApplicationServer solution or a WebFrontEnd solution, the important part is that it has an assembly marked to be deployed to the GAC:

http://schemas.microsoft.com/sharepoint/”>

…

The above solution, when deployed via either stsadm or the new PowerShell commandlets in SharePoint 2010 will attempt to put the assembly “MyAssembly.dll” into the GAC on every ApplicationServer in the farm. There are two possible reasons for this deployment to fail with the above error:

1) The assembly is already in the GAC. For the initial deploy, it appears the SharePoint uses “/i” and not “/if” and so it could fail if the assembly is already there. Most of the time, I find that this is not the issue but it could be for you. You can test if this is the issue by simply removing the version of the dll which is already there. If that is causing it, you need to figure out why you are trying to GAC the same version of the dll more than once.

2) You have not disabled strong name verification. For us … this was always the issue. Signing your assemblies is a time consuming process for a large product and so often you mark your assemblies for delay signing and then only sign “special” builds. In those cases, you need to disable strong name verification in order to add un-signed assemblies to the GAC. The tool for doing this is sn.exe which can be found in your Windows SDK directory. Using the –Vr option you can mark classes of assemblies to be skipped when it comes to verification. The classic command is “sn.exe –Vr *,*” which is indicating that any assembly with any public key token should be skipped. You might also chose to just skip verification for assemblies with your public key token with something like: “sn –Vr *,89845dcd8080cc91”. Whatever method you chose, there are a couple of key things you need to understand to make sure sn has “worked” for you:

Deploying a solution will deploy the solution to every appropriate machine in your farm. If you install an msi on MachineA in the farm and on that machine run sn.exe, but then deploy a solution and MachineB is also a member of the farm, you had to have run sn.exe on that machine also.
Sn.exe comes in a 32 and 64 bit flavor. It is difficult to tell which version you are running if someone just gave you the bits in a random folder (in the SDK folder the bits are under “architecturely” marked folder names). If you run the 32 version of sn.exe it will not help you when trying to GAC the binary on the 64-bit SharePoint server. There are a number of ways to try and figure out what is marked for “skipping” using the sn.exe tool, but I like to go directly to the registry because there is very little room for confusion there. Sn.exe is adding entries to the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\StrongName\Verification\ and HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\StrongName\Verification\ hives of the registry for whatever parameters you pass into the call to “sn.exe –Vr”. If you look in your registry and find that you only see an entry under the Wow6432Node, then you have only run the 32 bit version of sn. You will need to find the 64-bit version of sn.exe to get your skip registrations in the appropriate place (you can actually add them manually if you feel confident enough to edit the registry by hand).
THIS IS THE MOST OFTEN OVERLOOKED STEP – The list of what to skip when it comes to verification is actually cached. Normally, when you run gacutil.exe by hand, the process starts, loads the skip list, verifies, fails / succeeds and shuts down. Hence, if it fails and you then run sn.exe, the next time you run gacutil.exe it will succeed. This is *not* how it works with SharePoint. Solution deployment is done within the context of their timer service. This is an nt service which is not shutdown after a failed / successfull deployment of your solution. Hence, if your solution fails to deploy because of a strong name verification issue, you run “sn –Vr *,*” to enable skipping verification, and then try to deploy your solution again … *it will fail again*. You must recycle the SharePoint 2010 Timer service on every machine on which your solution is going to be deployed (specifically on which you just ran sn.exe) in order to get the skip list reloaded. I have seen people get frustrated and so reboot their machines which will obviously work, but all you really needed to do was recycle the timer service.

Hopefully this will help some SharePoint developers avoid wasting too much time figuring out why this was failing.

HTH
Lee

The data connection uses Windows Authentication and user credentials could not be delegated

powerpivotwahoo — Tue, 09 Feb 2010 00:30:15 +0000

This is one of the two main errors that users could see from Excel Services when using PowerPivot. This is encountered when refreshing PowerPivot data connections or performing an action which requires re-querying the PowerPivot database, such as clicking on a slicer or expanding a node in a pivot table. To debug, some level of understanding of what Excel Services is doing is required. For this, I recommend reading an earlier post on this blog by Dave … http://powerpivotgeek.com/2009/12/11/excel-services-delegation/. In general, this is an add-on to Dave’s post which is a quick summary of how to debug this error.

Here is a quick list of most likely causes for this error and how to debug and fix them (in order based on my experience of likelihood to be the root cause):

Is the Claims to Windows Token Service started on the server running Excel Calculation Service (ECS)? For detailed information on the Claims to Windows Token Service, you can read the msdn article at http://msdn.microsoft.com/en-us/library/ee517278.aspx. This service is turned on when doing a “New Farm” installation of PowerPivot, but if you do an “Existing Farm” installation, or have ECS running on a different machine, the Claims to Windows Token Service might not be started. You can validate this via the “Services on Server” option in SharePoint’s Central Administration web site. For more information on this and why you should not try to manage this service from Service Control Manager (SCM) refer to http://powerpivotgeek.com/2010/01/18/why-you-shouldnt-stop-start-analysis-services-from-scm-when-running-in-sharepoint-integration-mode/. Claims to Windows Token Service is a managed SharePoint service in SharePoint 2010 and so should only be started /stopped from SharePoint Central Administration. As well as simply managing the service, SharePoint configures the security permissions for this service automatically as part of their setup so that all SharePoint Shared Services (which includes ECS) can use it. If you find that the service is stopped on the machine running ECS, start it. After starting this service, you should not need to do any type of IisReset to see the system start working.
Is your machine connected to the network? Dave has written a good blog on this also (http://powerpivotgeek.com/2009/11/06/taking-your-server-off-the-network/). If you are actually trying to run a PowerPivot demo with a machine which is not on the network, follow the steps in Dave’s blog to configure ECS to use a set of stored credentials for the PowerPivot datasource. If you are not doing this on purpose, then connect back to the network.
The final cause would be that for some reason other than network connectivity, the Claims to Windows Token Service is not able to convert the SAML claims token to a Windows User security token. Dave provides a lot of details on these potential issues. Some quick questions to ask yourself:
- Is the client user account (the logged in user who is browsing the workbook in IE) a domain account? If the account is a local machine account, then the Claims to Windows Token service will not be able to retrieve a Windows user security token. We do not support this scenario in V1 of PowerPivot. Interactive users must be domain users. For demo purposes in a bind you might try the workaround Dave provided for taking the server off of the network, but I have not personally tested it in this case.
- Is the client user account in a different domain than the SharePoint servers? This is completely supported, but there must be a trust relationship established between the two domains. You could verify if a missing trust issue is causing your problems by logging in as a user account in the same domain as the SharePoint servers and try interacting with the workbooks (note that you had to have given that user access to the workbook). If it works for users in the same domain but not for users in other domains, it might be an issue with cross domain trust. Contact your domain admin to figure out what the relationships are setup as.
- What account is the Claims to Windows Token Service running as? By default it is configured to run as Local System, and I am not aware of the reasons for changing this configuration (the msdn article also refers to the fact that it should be running as Local System). While there might be a good reason for trying to change it, it is possible that the person who altered it did not understand the implications of this change. You should probably track down the person who changed it and get an understanding of why. If you have permissions, switch it back to Local System and try the scenario again. If it works, you will need to determine why it was changed in the first place.
- If you have gotten this far and none of the above have solved your issue, then there is the possibility that you have some custom AD configuration which is causing the issue. Dave points out one possibility:
  
  The account being used as the Excel Services service account must have AD rights to be able to query the object. One place where we know this restriction comes into play is if you have configured your domain controller to have a subgroup under “Users”, e.g. “Service Accounts”, which is a separate AD group that derives from “Users” –> but I am sure that there are more. AD rights for service accounts is a common problem across all of SharePoint.
  
  What Dave describes is one possibility where an AD configuration could cause this issue. As we discover more potential AD configurations that could cause this issue, we will try to update this list.

Hopefully this list will help you quickly debug and fix this issue on your system. HTH

Lee