Hi Kiran2015,
According to your description, you pinned Outlook 2010 item in taskbar, and after upgrading your office 2010 to office 2013, the Outlook 2010 didnt change and you wonder to remove it.
Windows Taskbar pinned items are stored in two locations:
- In the File System: Under %APPDATA%\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar, the items shortcuts are stored
- In the registry: Under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Taskband, the items pinning configuration is stored
Windows is then using registry keys to get the Taskbar pinning configuration and the File System to get the shortcuts. Missing one of them results in non-successful or incorrect pinning.
About how to remove pinned items from the Taskbar, this can be achieved by using Remove pinned programs from the Taskbar setting under User Configuration\Policies\Administrative Templates\Start Menu and Taskbar. This setting requires at least Windows 7 or
Windows Server 2008 R2.

Note: By enabling this setting, all pinned items will be removed. End users will not be able to do Taskbar pinning and the previously shared methods are no longer applicable.
Please read this article about managing Windows Taskbar Items pinning by Group Policy:
http://social.technet.microsoft.com/wiki/contents/articles/28239.how-to-manage-windows-taskbar-items-pinning-using-group-policy.aspx
From this article you can get how to add new Taskbar items pinning, replace it and remove it.
Hope its helpful.
Regards,