check if 2 columns are equal - Pandas

 After joining 2 dataframes you may have what appear to be redundant columns. It is tempting to be happy with a scan of the 1st few or last few columns but for very large datatsets you don't know what lies beneath. The pandas equals function is here to help. See code snippet below.


Comments

Post a Comment

Popular posts from this blog

Record of error messages when writing ML code

I'm an MBA turned data scientist who enjoys applying data science to real world problems. Of particular interest to me are capital markets and the telecom industry (my current employer).  While working on machine learning problems, I've often been stuck trying to decipher the long and scary error messages which make you wonder if something really bad has happened.  Oftentimes it is the result of an incorrect datatype or a missed comma or bracket. Google and Stackoverflow are usually helpful but it still takes a while to figure out the solution that really works for you.  So I've decided to keep a blog of error messages that I face when writing code and how I managed to solve them. I will be writing predominantly in python at this time. My other blogs are http://realworldml.substack.com for overall ML work including some general principles when working on machine learning projects as well as application of data science skills to specific domains. My other blog http:/...
Read more

Convert dataframe values to a list

Image
<class 'pandas.core.frame.DataFrame'> --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-5-18d68697e7bd> in <module> 1 a = df . quantile ( [ 0.25 , 0.50 , 0.75 ] ) ; 2 print ( type ( a ) ) ----> 3 a = a . tolist ( ) 4 print ( type ( a ) ) ~\.conda\envs\pybase\lib\site-packages\pandas\core\generic.py in __getattr__ (self, name) 5137 if self . _info_axis . _can_hold_identifiers_and_holds_name ( name ) : 5138 return self [ name ] -> 5139 return object . __getattribute__ ( self , name ) 5140 5141 def __setattr__ ( self , name : str , value ) -> None : AttributeError : 'DataFrame' object has no attribute 'tolist' Oftentimes we want to convert a dataframe or a column of a dataframe to a list. the pandas function tol...
Read more

TypeError: only list-like objects are allowed to be passed to isin(), you passed a [int]

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-31-29405b792f60> in <module> 2 for yr in yr_list [ : 2 ] : 3 print ( type ( yr ) ) ----> 4 temp_df = nifty_data [ nifty_data [ 'year' ] . isin ( yr ) ] 5 print ( temp_df . head ( ) ) ~\.conda\envs\py36\lib\site-packages\pandas\core\series.py in isin (self, values) 4683 Name : animal , dtype : bool 4684 """ -> 4685 result = algorithms . isin ( self , values ) 4686 return self._constructor(result, index=self.index).__finalize__( 4687 self , method = "isin" ~\.conda\envs\py36\lib\site-packages\pandas\core\algorithms.py in isin (comps, values) 416 if not is_list_like ( values ) : 417 raise TypeError( --> 418 "onl...
Read more